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Description 



Security System with Methodology for 
Interprocess Communication Control 

Cross Reference to Related Applications 

[0001] The present application is related to and claims the bene- 
fit of priority of the following commonly-owned, 
presently-pending provisional application(s): application 
serial no. 60/320,079 (Docket No. VIV/0011.00), filed 
April 1, 2003, entitled "Security System with Methodology 
for Interprocess Communication Control", of which the 
present application is a non-provisional application 
thereof. The present application is also related to the fol- 
lowing commonly-owned, presently-pending applica- 
tion(s): application serial no. 10/159,820 (Docket No. 
VIV/0005.01), filed May 31, 2002, entitled "System and 
Methodology for Security Policy Arbitration"; application 
serial no. 10/249,803 (Docket No. VIV/0008.01), filed 
May 8, 2003, entitled "Security System And Methodology 
For Providing Indirect Access Control". The disclosures of 



each of the foregoing applications are hereby incorpo- 
rated by reference in their entirety, including any appen- 
dices or attachments thereof, for all purposes. 
Copyright Statement 

[0002] a portion of the disclosure of this patent document con- 
tains material, which is subject to copyright protection. 
The copyright owner has no objection to the facsimile re- 
production by anyone of the patent document or the 
patent disclosure as it appears in the Patent and Trade- 
mark Office patent file or records, but otherwise reserves 
all copyright rights whatsoever. 
Appendix Data 

[0003] Computer Program Listing Appendix under Sec. 1.52(e): 
This application includes a transmittal under 37 C.F.R. 
Sec. 1.52(e) of a Computer Program Listing Appendix. The 
Appendix, which comprises text file(s) that are IBM-PC 
machine and Microsoft Windows Operating System com- 
patible, includes the below-listed file(s). All of the mate- 
rial disclosed in the Computer Program Listing Appendix 
can be found at the U.S. Patent and Trademark Office 
archives and is hereby incorporated by reference into the 
present application. 



[0004] object Description: SourceCode.txt, created: 7/25/03 
2:21pm, size: 22 KB; Object ID: File No. 1; Object Con- 
tents: Source Code. 
Background of Invention 

[0005] i. Field of the Invention 

[0006] The present invention relates generally to systems and 
methods for maintaining security of computer systems 
connected to one or more networks (Local Area Networks 
or Wide Area Networks) and, more particularly, to a secu- 
rity system with methodology for interprocess communi- 
cation control. 

[0007] 2. Description of the Background Art 

[0008] jhe first computers were largely stand-alone units with 

no direct connection to other computers or computer net- 
works. Data exchanges between computers were mainly 
accomplished by exchanging magnetic or optical media 
such as floppy disks. Over time, more and more comput- 
ers were connected to each other using Local Area Net- 
works or LANs. In both cases, maintaining security and 
controlling what information a computer user could ac- 
cess was relatively simple because the overall computing 
environment was limited and clearly defined. 



[0009] with the ever-increasing popularity of the Internet, how- 
ever, more and more computers are connected to larger 
networks. Providing access to vast stores of information, 
the Internet is typically accessed by users through Web 
"browsers" (e.g., Microsoft® Internet Explorer or Netscape 
Navigator) or other Internet applications. Browsers and 
other Internet applications include the ability to access a 
URL (Universal Resource Locator) or "Web" site. In the last 
several years, the Internet has become pervasive and is 
used not only by corporations, but also by a large number 
of small businesses and individual users for a wide range 
of purposes. 

[0010] As more and more computers are now connected to the 
Internet, either directly (e.g., over a dial-up or broadband 
connection with an Internet Service Provider or ISP) or 
through a gateway between a LAN and the Internet, a 
whole new set of challenges face LAN administrators and 
individual users alike: these previously closed computing 
environments are now open to a worldwide network of 
computer systems. A particular set of challenges involves 
attacks by perpetrators (hackers) capable of damaging the 
local computer systems, misusing those systems, and/or 
stealing proprietary data and programs. 



[0011] The software industry has, in response, introduced a 
number of products and technologies to address and 
minimize these threats, including firewalls, proxy servers, 
and similar technologies — all designed to keep outside 
hackers from penetrating a computer system or corporate 
network. Firewalls are applications that intercept the data 
traffic at the gateway to a Wide Area Network (WAN) and 
check the data packets (i.e., Internet Protocol or "IP" pack- 
ets) being exchanged for suspicious or unwanted activi- 
ties. 

[0012] Another security measure that has been utilized by many 
users is to install an end point security (or personal fire- 
wall) product on a computer system to control traffic into 
and out of the system. An end point security product can 
regulate all traffic into and out of a particular computer. 
For example, an end point security product may permit 
specific "trusted" applications to access the Internet while 
denying access to other applications on a user's com- 
puter. To a large extent, restricting access to "trusted" ap- 
plications is an effective security method. However, there 
are cases in which an untrusted or malicious application 
may cause the trusted application to perform unautho- 
rized actions on its behalf, thereby circumventing current 



security mechanisms. 
[0013] | n present-day operating systems, such as Microsoft® 

Windows for example, a lot of interaction occurs between 
different processes (e.g., applications, drivers, and the 
like) which are running on a computer system at the same 
time. Moreover, considerable interaction occurs between 
different applications and so-called "services." A service 
may be thought of as a special case application that exists 
to serve other applications (e.g., by providing special 
functionality or performing particular tasks). For example, 
a DNS service is provided by a special application that 
performs DNS lookup services on behalf of other applica- 
tions. 

[0014] Notwithstanding the fact that a given computer system 
may be protected by a firewall or an end point security 
product, these services pose an additional security risk. 
Stated more generally, interprocess communication pro- 
vides additional opportunities for breaching security mea- 
sures employed to protect computer systems. For exam- 
ple, a rogue application can circumvent conventional se- 
curity measures by using services or interprocess commu- 
nication to cause another application to perform actions 
on its behalf. The rogue application uses services or inter- 



process communication as a proxy to obtain, in effect, an 
elevation of its security privileges. This elevation of its se- 
curity privileges enables it to breach security by causing 
another application or service to perform actions that the 
rogue application is not able to do itself (according to op- 
erating system privilege settings). In addition, the rogue 
application is able to disguise the fact that it is accessing 
the Internet by going through another application or ser- 
vice (e.g., an operating system service) in a manner that is 
not detected by a conventional firewall. 
[0015] For example, Windows XP includes a DNS (domain name 
system) service that performs DNS lookup on behalf of 
other applications. DNS is itself normally a harmless pro- 
tocol that contacts a DNS server for translating domain 
names (e.g., cnn.com) into IP addresses. However, a mali- 
cious application has the ability to use Windows' built-in 
DNS service to communicate with a malicious DNS server. 
For example, the malicious application may use the DNS 
service for a DNS lookup of "MySecret.Hacker.com". The 
DNS server at the hacker site ("Hacker.com") would then 
get a query from the local DNS server asking whether it 
has an IP address for "MySecret". In fact, what the hacker 
site DNS server receives is a token (string of "MySecret"). 



At this point, the malicious application may engage in al- 
most unlimited communication with the malicious DNS 
server using an awkward, but also very effective, protocol. 
[0016] This example of a malicious application using the DNS 

service to communicate with a malicious DNS server is il- 
lustrated in the diagram shown in Fig. 1A. As shown, on 
client machine 10, malicious application ("malware") 13 
communicates with a local DNS service 15 to perform a 
DNS look-up of "MySecret.Hacker.com" (where, in this ex- 
ample, "Hacker.com" is a malicious DNS server at a remote 
site). In response to this request, the local DNS service 15 
sends a request over the Internet to the malicious DNS 
server 18 asking whether the Hacker.com DNS server has 
an IP address for "MySecret". Upon receipt of this request, 
the malicious DNS server 18 has the token containing 
confidential information, "MySecret", from the client ma- 
chine 10. A conventional client-side firewall (e.g., applica- 
tion-oriented firewall software 11) would not block this 
transmission of confidential information as it only looks to 
see whether the malware application 13 was communicat- 
ing (directly) with the Internet. Since the foregoing secu- 
rity breach does not involve direct communication be- 
tween a potentially malicious application and the Internet, 



a conventional firewall would not detect the security 
breach. In other words, since the malicious application 
was able to masquerade its Internet access by going 
through an operating system service, the malicious appli- 
cation was able to breach security in a manner that would 
not be detected by a conventional firewall. 

[0017] The malicious application may also use some of the same 
approaches to attack a computer's underlying security ap- 
plication itself. For instance, the malicious application may 
use interprocess communications with another application 
to attack the security application, or the malicious appli- 
cation may attack the security application directly by 
posting user input messages (e.g., keystrokes, mouse in- 
put, or the like) to the security application. If the mali- 
cious application can disable the security application, it 
can gain unfettered access to the entire computer system. 

[0018] Referring again to Fig. 1A, the client machine 10 includes 
application-oriented firewall (end point security) software 
11, which serves to monitor potentially malicious applica- 
tions (e.g., malware 13) for unauthorized Internet access. 
For environments which allow one application to send 
messages to another application (e.g., Microsoft Windows 
environments), malware 13 may send messages to firewall 



software 11 in an attempt to disable the firewall. As illus- 
trated by an interprocess communication path 17, mal- 
ware 13 may send keystroke and/or mouse input mes- 
sages (e.g., using a Microsoft Windows "SendMessage" API 
call) that masquerade as user input. For example, when 
malware 13 attempts to access the Internet, assume that 
the firewall software 11 displays a dialog box to the user 
inquiring whether malware 13 should be allowed to access 
the Internet. At this point, malware 13 could masquerade 
as the user by sending forged user input (e.g., keystroke 
and/or mouse input messages) to the firewall software 11, 
via the interprocess communication path 17, thereby ef- 
fectively overcoming the security provided by firewall 
software 11. This is an instance of a malicious application 
using interprocess communications to either abuse/mis- 
use a service or shut down or interfere with a service that 
is critical for security. 
[0019] one current approach for preventing unauthorized com- 
munications is to attempt to obfuscate or "hide" applica- 
tion interfaces from other (potentially malicious) applica- 
tions. For example, random names can be assigned to 
windows, the appearance of items and title prompts can 
be changed, and so forth so that these interfaces are more 



difficult for a malicious application to locate and misuse. 
However, this approach is only of limited utility and does 
not fully solve the problem. 

[0020] Current systems also provide facilities for users to estab- 
lish access privileges. However, this solution is difficult to 
implement as considerable expertise is required and the 
facilities provided by certain operating systems are sub- 
ject to well-known weaknesses and vulnerabilities. It is 
very difficult to configure access privileges in a manner 
which provides security but at the same time does not in- 
terfere with the users' ability to perform normal business 
and/or personal activities. In addition, not all operating 
systems provide support for establishing access privi- 
leges. For example, personal editions of certain operating 
systems do not include this type of access permission set- 
ting feature. In fact, a considerable number of current op- 
erating systems (e.g., personal edition operating system 
versions) broadly permit applications to communicate with 
each other with few limitations. For these reasons, the 
permission setting features of current systems do not 
provide adequate protection against misuse of interpro- 
cess communications. 

[0021] a security solution is required that controls communica- 



tion channels among processes (including applications, 
services, and the like) to provide improved security. The 
solution should not only address the problem of a mali- 
cious application misusing another application or service, 
but should also provide a solution to the more general 
problem of using interprocess communications to breach 
security. The present invention provides a solution for 
these and other needs. 
Summary of Invention 

[0022] a security system with methodology for interprocess 

communication control is described. In one embodiment, 
a method for controlling interprocess communication is 
provided that includes steps of: defining rules indicating 
which system services a given application can invoke; 
trapping an attempt by a particular application to invoke a 
particular system service; identifying the particular appli- 
cation that is attempting to invoke the particular system 
service; and based on identity of the particular application 
and on the rules indicating which system services a given 
application can invoke, blocking the attempt when the 
rules indicate that the particular application cannot invoke 
the particular system service. 

[0023] | n another embodiment a method for regulating commu- 



nications between processes in a computer system com- 
prises: defining a policy specifying whether one process 
may communicate with another process; intercepting an 
attempt by a first process to communicate with a second 
process; identifying the first process that is attempting to 
communicate with the second process; identifying the 
second process; based on said policy, determining 
whether the first process may communicate with the sec- 
ond process; and allowing the first process to communi- 
cate with the second process if said policy indicates that 
the first process may communicate with the second pro- 
cess. 

[0024] | n another embodiment, a method for controlling inter- 
process communications from one application to another 
comprises: registering a first application to be protected 
from other applications; detecting an attempt to access 
the first application using interprocess communication; 
identifying a second application that is attempting to ac- 
cess the first application using interprocess communica- 
tion; and rerouting the attempt to access the first applica- 
tion through an interprocess communication controller 
that determines whether to allow the attempt, based on 
rules indicating whether the second application may ac- 



cess the first application using interprocess communica- 
tion. 

[0025] | n another embodiment, a system for regulating interpro- 
cess communication between applications is described. 
The system comprises a policy specifying applications that 
are permitted to communicate with a first application us- 
ing interprocess communication; a module for detecting a 
second application attempting to communicate with the 
first application using interprocess communication; and 
an interprocess communication controller for identifying 
the second application attempting to communicate with 
the first application and determining whether to permit 
the communication based upon the identification of the 
second application and the policy specifying applications 
permitted to communicate with the first application. 
Brief Description of Drawings 

[0026] pig. 1A is a block diagram illustrating an example of a 

malicious application using a DNS service to communicate 
with a malicious DNS server. 

[0027] pig. IB is a very general block diagram of an IBM- 
compatible computer system which may be used for im- 
plementing the present invention. 

[0028] Fig. 2 is a block diagram of a software system for control- 



ling the operation of the computer system of Fig. IB. 

[0029] pig. 3 is a high-level block diagram illustrating an envi- 
ronment in which the present invention may be embodied. 

[0030] pig. 4 presents a flowchart illustrating the high-level 

methods of operation of the system of the present inven- 
tion in controlling indirect access to networks and re- 
sources by a potentially malicious application. 

[0031] Fig. 5 presents a flowchart illustrating the high-level 

methods of operation of the system of the present inven- 
tion in regulating interprocess communications between 
applications. 

[0032] Fig. 6 is a block diagram illustrating the components of 
the Interprocess Communication Controller (IPCC). 
Detailed Description 

Glossary 

[0033] The following definitions are offered for purposes of illus- 
tration, not limitation, in order to assist with understand- 
ing the discussion that follows. 

[0034] DNS: The domain name system (DNS) is the way that In- 
ternet domain names are located and translated into In- 
ternet Protocol addresses. A domain name is a meaningful 
and easy-to-remember "handle" for an Internet address. 



[0035] End point security: End point security is a way of manag- 
ing and enforcing security on each computer instead of 
relying upon a remote firewall or a remote gateway to 
provide security for the local machine or environment. End 
point security involves a security agent that resides locally 
on each machine. This agent monitors and controls the 
interaction of the local machine with other machines and 
devices that are connected on a local area network (LAN) 
or a larger wide area network (WAN) such as the Internet 
in order to provide security to the machine. For further in- 
formation regarding an end point security solution for 
controlling the interaction of a machine with other con- 
nected machines and devices, see e.g., commonly-owned 
U.S. Patent No. 5,987,611, entitled "System and Method- 
ology for Managing Internet Access on a per Application 
Basis for Client Computers Connected to the Internet", the 
disclosure of which is hereby incorporated by reference. 

[0036] Firewall: A firewall is a set of related programs, typically 
located at a network gateway server, that protects the re- 
sources of a private network from other networks by con- 
trolling access into and out of the private network. (The 
term also implies the security policy that is used with the 
programs.) A firewall, working closely with a router pro- 



gram, examines each network packet to determine 
whether to forward it toward its destination. A firewall 
may also include or work with a proxy server that makes 
network requests on behalf of users. A firewall is often in- 
stalled in a specially designated computer separate from 
the rest of the network so that no incoming request di- 
rectly accesses private network resources. 
[0037] LPC: LPC re f ers t0 Loca | Procedure Call (LPC), a facility of 
the Windows NT/2000/XP operating system which is de- 
signed for subsystem communication. LPC is comparable 
to the Remote Procedure Call (RPC) used in UNIX environ- 
ments for communication between processes running on 
two different machines; however, LPC has been optimized 
for communication between processes running on the 
same machine. The LPC facility is used as a communica- 
tion mechanism between system services (or subsystems) 
and their client processes. A client thread invokes LPC 
when it needs some service from the subsystem. The LPC 
mechanism passes on the parameters for the service invo- 
cation to the server thread. The server thread executes the 
service and passes the results back to the client thread 
using the LPC facility. For further information regarding 
LPC, see e.g., Dabak, Prasad et al, "Undocumented Win 



NT", October 1989, M&T Books, the disclosure of which is 
hereby incorporated by reference. An on-line copy of this 
book is currently available via the Internet at 
www.windowsitlibrary.com. 

[0038] Network: A network is a group of two or more systems 
linked together. There are many types of computer net- 
works, including local area networks (LANs), virtual private 
networks (VPNs), metropolitan area networks (MANs), 
campus area networks (CANs), and wide area networks 
(WANs) including the Internet. As used herein, the term 
"network" refers broadly to any group of two or more 
computer systems or devices that are linked together 
from time to time (or permanently). 

[0039] Process: A process or task refers to the combination of a 
program (e.g., an application program) being executed on 
an operating system and associated bookkeeping infor- 
mation used by the operating system. When a program is 
executed, the operating system typically creates a new 
process for each instance of the program being executed. 
The process is like an envelope for the program which 
identifies the program with a process number (e.g., a pro- 
cess identifier or "ID") and associates other bookkeeping 
information to the process. Many operating systems, in- 



eluding UNIX and Windows, are capable of running many 
processes (or tasks) at the same time and are called 
multi-tasking operating systems. 

[0040] Security policy: In general terms, a security policy is an 
organization's statement defining the rules and practices 
that regulate how it will provide security, handle intru- 
sions, and recover from damage caused by security 
breaches. An explicit and well-defined security policy in- 
cludes a set of rules that are used to determine whether a 
given subject will be permitted to gain access to a specific 
object. A security policy may be enforced by hardware and 
software systems that effectively implement access rules 
for access to systems and information. Further informa- 
tion on security policies is available in "RFC 2196: Site Se- 
curity Handbook, (September 1997)," the disclosure of 
which is hereby incorporated by reference. In this docu- 
ment, "security policy" or "policy" refers to a set of secu- 
rity policies and rules employed by an individual or by a 
corporation, government entity, or any other organization 
operating a network or other computing resources. 

[0041] Service: A service is a special type of application that 

serves other applications by providing special functional- 
ity or performing particular tasks on behalf of the other 



application. Most modern operating systems typically in- 
clude subsystems which make available a number of dif- 
ferent services to applications which are running on the 
operating system. In the Microsoft Windows environment, 
for example, the operating system provides a LPC (local 
procedure call) facility which enables efficient communi- 
cation with subsystems providing services. An example of 
a service is a DNS service provided by a subsystem of the 
Windows operating system subsystem that performs DNS 
lookup services on behalf of another application. For fur- 
ther information regarding services, see e.g., Dabak, 
Prasad et al, "Undocumented Win NT", October 1989, M&T 
Books, the disclosure of which is hereby incorporated by 
reference. An on-line copy of this book is currently avail- 
able via the Internet at www.windowsitlibrary.com. 
[0042] winsock: Winsock refers to the Microsoft Windows Sockets 
2 interface, which enables programmers to create net- 
work-capable applications to transmit application data 
across a network independent of the network protocol be- 
ing used. Winsock defines a standard service provider in- 
terface (SPI) between the application programming inter- 
face (API), with its exported functions and the protocol 
stacks. It uses the sockets paradigm that was first popu- 



larized by Berkeley Software Distribution (BSD) UNIX. For 
further information regarding Winsock, see e.g., "Windows 
Sockets API Reference", available from Microsoft Corpora- 
tion, the disclosure of which is hereby incorporated by 
reference. A copy of this documentation is currently avail- 
able via the Internet at msdn.microsoft.com/library. 
Introduction 

[0043] Referring to the figures, exemplary embodiments of the 

invention will now be described. The following description 
will focus on the presently preferred embodiment of the 
present invention, which is implemented in desktop and/ 
or server software (e.g., driver, application, or the like) 
operating in an Internet-connected environment running 
under an operating system, such as the Microsoft Win- 
dows operating system. The present invention, however, 
is not limited to any one particular application or any par- 
ticular environment. Instead, those skilled in the art will 
find that the system and methods of the present invention 
may be advantageously embodied on a variety of different 
platforms, including Macintosh, Linux, BeOS, Solaris, 
UNIX, NextStep, FreeBSD, and the like. Therefore, the de- 
scription of the exemplary embodiments that follows is 
for purposes of illustration and not limitation. The exem- 



plary embodiments are primarily described with reference 
to block diagrams or flowcharts. As to the flowcharts, 
each block within the flowcharts represents both a 
method step and an apparatus element for performing the 
method step. Depending upon the implementation, the 
corresponding apparatus element may be configured in 
hardware, software, firmware or combinations thereof. 
Computer-based implementation 

[0044] Basic system hardware (e.g., for desktop and server computers) 

[0045] The present invention may be implemented on a conven- 
tional or general-purpose computer system, such as an 
IBM-compatible personal computer (PC) or server com- 
puter. Fig. IB is a very general block diagram of an IBM- 
compatible system 100. As shown, system 100 comprises 
a central processing unit(s) (CPU) or processor(s) 101 cou- 
pled to a random-access memory (RAM) 102, a read-only 
memory (ROM) 103, a keyboard 106, a printer 107, a 
pointing device 108, a display or video adapter 104 con- 
nected to a display device 105, a removable (mass) stor- 
age device 115 (e.g., floppy disk, CD-ROM, CD-R, CD-RW, 
DVD, or the like), a fixed (mass) storage device 116 (e.g., 
hard disk), a communication (COMM) port(s) or inter- 



face(s) 110, a modem 112, and a network interface card 
(NIC) or controller 111 (e.g., Ethernet). Although not 
shown separately, a real time system clock is included 
with the system 100, in a conventional manner. 
[0046] CPU 101 comprises a processor of the Intel Pentium family 
of microprocessors. However, any other suitable proces- 
sor may be utilized for implementing the present inven- 
tion. The CPU 101 communicates with other components 
of the system via a bi-directional system bus (including 
any necessary input/output (I/O) controller circuitry and 
other "glue" logic). The bus, which includes address lines 
for addressing system memory, provides data transfer be- 
tween and among the various components. Description of 
Pentium-class microprocessors and their instruction set, 
bus architecture, and control lines is available from Intel 
Corporation of Santa Clara, CA. Random-access memory 
102 serves as the working memory for the CPU 101. In a 
typical configuration, RAM of sixty-four megabytes or 
more is employed. More or less memory may be used 
without departing from the scope of the present inven- 
tion. The read-only memory (ROM) 103 contains the basic 
input/output system code (BIOS) — a set of low-level rou- 
tines in the ROM that application programs and the oper- 



ating systems can use to interact with the hardware, in- 
cluding reading characters from the keyboard, outputting 
characters to printers, and so forth. 
[0047] M ass storage devices 115, 116 provide persistent storage 
on fixed and removable media, such as magnetic, optical 
or magnetic-optical storage systems, flash memory, or 
any other available mass storage technology. The mass 
storage may be shared on a network, or it may be a dedi- 
cated mass storage. As shown in Fig. IB, fixed storage 
116 stores a body of program and data for directing oper- 
ation of the computer system, including an operating sys- 
tem, user application programs, driver and other support 
files, as well as other data files of all sorts. Typically, the 
fixed storage 116 serves as the main hard disk for the 
system. 

[0048] | n basic operation, program logic (including that which 
implements methodology of the present invention de- 
scribed below) is loaded from the removable storage 115 
or fixed storage 116 into the main (RAM) memory 102, for 
execution by the CPU 101. During operation of the pro- 
gram logic, the system 100 accepts user input from a 
keyboard 106 and pointing device 108, as well as speech- 
based input from a voice recognition system (not shown). 



The keyboard 106 permits selection of application pro- 
grams, entry of keyboard -based input or data, and selec- 
tion and manipulation of individual data objects displayed 
on the screen or display device 105. Likewise, the pointing 
device 108, such as a mouse, track ball, pen device, or the 
like, permits selection and manipulation of objects on the 
display device. In this manner, these input devices sup- 
port manual user input for any process running on the 
system. 

[0049] The computer system 100 displays text and/or graphic 
images and other data on the display device 105. The 
video adapter 104, which is interposed between the dis- 
play 105 and the system's bus, drives the display device 
105. The video adapter 104, which includes video memory 
accessible to the CPU 101, provides circuitry that converts 
pixel data stored in the video memory to a raster signal 
suitable for use by a cathode ray tube (CRT) raster or liq- 
uid crystal display (LCD) monitor. A hard copy of the dis- 
played information, or other information within the sys- 
tem 100, may be obtained from the printer 107, or other 
output device. Printer 107 may include, for instance, an 
HP LaserJet printer (available from Hewlett Packard of Palo 
Alto, CA), for creating hard copy images of output of the 



system. 

[0050] The system itself communicates with other devices (e.g., 
other computers) via the network interface card (NIC) 111 
connected to a network (e.g., Ethernet network, Bluetooth 
wireless network, or the like), and/or modem 112 (e.g., 
56K baud, ISDN, DSL, or cable modem), examples of 
which are available from 3Com of Santa Clara, CA. The 
system 100 may also communicate with local occasion- 
ally-connected devices (e.g., serial cable-linked devices) 
via the communication (COMM) interface 110, which may 
include a RS-232 serial port, a Universal Serial Bus (USB) 
interface, or the like. Devices that will be commonly con- 
nected locally to the interface 110 include laptop comput- 
ers, handheld organizers, digital cameras, and the like. 

[0051] IBM-compatible personal computers and server computers 
are available from a variety of vendors. Representative 
vendors include Dell Computers of Round Rock, TX, 
Hewlett-Packard of Palo Alto, CA, and IBM of Armonk, NY. 
Other suitable computers include Apple-compatible com- 
puters (e.g., Macintosh), which are available from Apple 
Computer of Cupertino, CA, and Sun Solaris workstations, 
which are available from Sun Microsystems of Mountain 
View, CA. 



[0052] Basic system software 

[0053] illustrated in Fig. 2, a computer software system 200 is 

provided for directing the operation of the computer sys- 
tem 100. Software system 200, which is stored in system 
memory (RAM) 102 and on fixed storage (e.g., hard disk) 
116, includes a kernel or operating system (OS) 210. The 
OS 210 manages low-level aspects of computer operation, 
including managing execution of processes, memory allo- 
cation, file input and output (I/O), and device I/O. One or 
more application programs, such as client application 
software or "programs" 201 (e.g., 201a, 201b, 201c, 
20 Id) may be "loaded" (i.e., transferred from fixed stor- 
age 116 into memory 102) for execution by the system 
100. The applications or other software intended for use 
on the computer system 100 may also be stored as a set 
of downloadable computer-executable instructions, for 
example, for downloading and installation from an Inter- 
net location (e.g., Web server). 

[0054] System 200 includes a graphical user interface (GUI) 215, 
for receiving user commands and data in a graphical (e.g., 
"point-and-click") fashion. These inputs, in turn, may be 
acted upon by the system 100 in accordance with instruc- 
tions from operating system 210, and/or client applica- 



tion module(s) 201. The GUI 215 also serves to display the 
results of operation from the OS 210 and application(s) 
201, whereupon the user may supply additional inputs or 
terminate the session. Typically, the OS 210 operates in 
conjunction with device drivers 220 (e.g., "Winsock" driver 
— Windows' implementation of a TCP/IP stack) and the 
system BIOS microcode 230 (i.e., ROM-based microcode), 
particularly when interfacing with peripheral devices. OS 
210 can be provided by a conventional operating system, 
such as Microsoft Windows 9x, Microsoft Windows NT, Mi- 
crosoft Windows 2000, or Microsoft Windows XP, all avail- 
able from Microsoft Corporation of Redmond, WA. Alter- 
natively, OS 210 can also be an alternative operating sys- 
tem, such as the previously mentioned operating systems. 
[0055] The above-described computer hardware and software are 
presented for purposes of illustrating the basic underlying 
desktop and server computer components that may be 
employed for implementing the present invention. For 
purposes of discussion, the following description will 
present examples of interprocess communications in a 
Windows operating system environment. The present in- 
vention, however, is not limited to any particular environ- 
ment or device configuration. In particular, a Windows op- 



erating system environment is not necessary to the inven- 
tion, but is used to provide a framework for discussion. 
Instead, the present invention may be implemented in any 
type of system architecture or processing environment ca- 
pable of supporting the methodologies of the present in- 
vention presented in detail below. 
Overview 

[0056] | n accordance with the present invention, a security sys- 
tem is provided that controls communication channels 
between and among applications and services. Here, a 
"service" may be considered as a special type of applica- 
tion that provides particular functionality or performs 
specific tasks on behalf of another application. In the Mi- 
crosoft Windows environment, for example, the operating 
system provides a LPC (local procedure call) facility that 
underlies a lot of higher-level operating system services. 
The LPC facility enables efficient communication with sub- 
systems of the operating system which provide various 
operating system services. As another example from the 
Microsoft Windows environment, processes may commu- 
nicate using DDE (dynamic data exchange), which itself 
relies on Windows messaging. By controlling the various 
different communication channels amongst applications 



and services, the present invention is able to provide im- 
proved security. 

[0057] Normally in an operating system, the various applications 
and services "live" (reside) in separate memory spaces. In 
order for one application to communicate with another 
application or service, the operating system must provide 
some sort of communication channel between the two ap- 
plications or services. In UNIX operating environments, for 
example, the operating system provides "named pipes". In 
Microsoft Windows environments, the operating system 
provides "ports" which are similar to named pipes. Mi- 
crosoft Windows also provides a simpler communication 
channel using Windows messaging, which allows one ap- 
plication to send a message to a specific window or thread 
of another application. 

[0058] | n order to control direct and indirect access by an appli- 
cation to a network (e.g., the Internet), cases involving 
both direct and indirect access need to be addressed. In 
order to control instances of direct access by an applica- 
tion, it is relatively straightforward to match activity/traf- 
fic against an established security policy. When exceptions 
to the policy occur either the user may be asked for per- 
mission (e.g., via presentation of a dialog box in the user 



interface) or default handling may be applied (e.g., deny 
access). In instances of indirect access, however, a differ- 
ent approach must be applied. In order to control indirect 
access, the security system of the present invention inter- 
cepts the operating system call(s) that allows an applica- 
tion (e.g., a potentially malicious application) to subscribe 
to or otherwise access one of the communication channels 
(e.g., a "port" providing access to a particular operating 
system service). A policy is then applied against the appli- 
cation attempting to access a service (in a manner similar 
to that done for direct access attempts) based upon the 
service that the application is attempting to access. For 
example, the security system of the present invention 
treats, for purposes of security, the activity of talking to 
the DNS service the same as attempting to directly access 
the Internet. 

[0059] | n particular, to control indirect access certain communi- 
cations (e.g., "open port" local procedure calls in Windows 
environments) are intercepted at the level of the operating 
system kernel. At that point, the system of the present in- 
vention can determine what address or port a given appli- 
cation is attempting to talk to, based on a text string 
passed during invocation of the communication (e.g., 



"open port") call. Based on the determined address or 
port, the system of the present invention can uncover an 
attempt to access the Internet. Thus, in a situation in 
which a given application is attempting to access a local 
DNS service, the system may treat the attempt to access 
this service as the equivalent of a direct DNS lookup and 
apply the appropriate security policy accordingly. 
[0060] a first aspect of the present invention provides the ability 
to control access to well-known services by potentially 
malicious applications. Malicious applications may seek to 
exploit well-known services that exist on a system, such 
as those that are published by services components or by 
the operating system. In operation, a service component, 
such as an RPC-based DNS resolver, may create a "sub- 
scription" address or "port" that a potentially malicious 
application may use to talk to the service. The created 
subscription address or port serves as a mechanism that 
allows a potentially malicious application to indirectly 
communicate with other applications and resources (e.g., 
with a malicious server) using specifically formatted mes- 
sages. Therefore, in accordance with the present inven- 
tion, an application's ability to engage in such communi- 
cation is regulated for the purpose of enforcing security 



policies. 

[0061] Control is achieved by filtering an application's request to 
access a service through security policy management 
mechanisms of the present invention in order to deter- 
mine whether or not the application should be allowed to 
access the service. This approach is adopted since one of 
the end results of the application's access to the service is 
the application obtaining access to particular resources or 
networks (e.g., the Internet). In particular, the approach 
controls the ability of an untrusted application to access 
the Internet via indirect means, such as through a trusted 
application (which is operating in response to the un- 
trusted application). 

[0062] As another aspect of the present invention, policy infor- 
mation describing different protection levels for applica- 
tions is registered with the operating system (kernel). This 
approach is used to control the ability of one application 
to send messages (e.g., user input-based messages, such 
as keystrokes, mouse input, or the like) to another appli- 
cation. For example, client-side firewall management 
software (e.g., a ZoneAlarm® product or process) can be 
registered with the operating system and assigned a pro- 
tection level that prevents other applications from sending 



or posting certain types of messages (e.g., keyboard mes- 
sages and mouse input messages) to the firewall (security) 
software. 

[0063] Different mechanisms may be applied to protect an appli- 
cation from communications sent by other applications. 
For instance, an application may be registered with the 
system and specific rules provided that must be followed 
to communicate with the application. Alternatively, an ap- 
plication may be unregistered in which case it must ask 
for permission in order to engage in certain activities. The 
system responds based on policy rules for an untrusted 
(unregistered) application. 
System components 

[0064] pig. 3 is a high-level block diagram illustrating an envi- 
ronment 300 in which the present invention may be em- 
bodied. "Malware" 321 represents an unknown application 
that is potentially malicious. This unknown malware appli- 
cation 321 will use various aspects of the operating sys- 
tem. In the Microsoft Windows environment, for example, 
the malware application would interface with "kernel32" 
and "user32" components (shown at 323), which are oper- 
ating system components that provide application pro- 
gramming interfaces (APIs) to applications. As shown in 



Fig. 3, the kernel components 323 exist at the user 
(privilege) level (ring 3). These components communicate 
with corresponding lower-level (i.e., more privileged) op- 
erating system components, Win32 kernel and NTOS ker- 
nel components 325, that exist at the kernel or root 
(privilege) level (ring 0). The NTOS kernel component pro- 
vides basic, low-level operating services, such as file sys- 
tem management. The Win32 kernel component supports 
other low-level subsystems, such as the graphics subsys- 
tem (e.g., Windows GDI subsystem). 
[0065] The built-in security (i.e., privilege level enforcement) 

provided by the operating system prevents a process from 
crossing over from the "user" privilege level (ring 3) to the 
"kernel" privilege level (ring 0) unless proper authorization 
is obtained. To cross this line, a process must either be 
part of the operating system itself or be a process ac- 
corded kernel-level privileges, such as a low-level device 
driver (e.g., either digitally signed by an authorized entity, 
or approved by an administrator with root privileges). In 
the configuration shown in Fig. 3, malware 321 is at- 
tempting to access a service, such as DNS (or other) ser- 
vice 331, which exists at the user (privilege) level. Access 
is ordinarily achieved by invoking the corresponding ker- 



nel-level local procedure call (LPC), for example, a LPC 
DNS 333. 

[0066] | n accordance with the present invention, security service 
components with interprocess controlling capability are 
provided, including a kernel component and two user- 
level components. The user-level components include a 
TrueVector® engine (VS_MON) 343 and a firewall manager 
(ZoneAlarm®) 341. The TrueVector engine 343 provides 
policy enforcement of the firewall. The firewall manager 
341 provides user interaction (e.g., user interface) for the 
TrueVector engine 343. The kernel component comprises 
the TrueVector device driver (VS_DATA_NT) 345 which in- 
cludes the core firewall functionality. Of particular interest 
to the present invention is the addition of an Interprocess 
Communication Controller (IPCC) subcomponent 349 to 
the TrueVector device driver 345, which is described in 
further detail below. 

[0067] | n ordinary operation, a service would publish a port by 
invoking a corresponding operating system API call for 
opening a port (e.g., a "NtCreatePort" API call in a Win- 
dows environment). The port, which is an abstraction of a 
communication channel, is associated with a text string 
that identifies that specific port. The operating system 



kernel component 325 maintains a list of open ports. 
During its attempt to gain unauthorized Internet access, 
malware 321 attempts to create a communication channel 
(e.g., port) to the DNS service 331 and then connect to 
that port. More specifically, malware 321 invokes the Win- 
dows Winsock (Windows socket) API (residing in user-level 
components 323 at ring 3) which in turn invokes the 
lower-level kernel components 325, specifically invoking 
the NTOS component operating at ring 0. The incoming 
connection request of malware 321 that attempts to cre- 
ate a communication channel can be trapped and re- 
directed. The connection request attempt also includes 
sufficient information (e.g., text string) to allow one to 
determine the type of service that is being requested. In- 
vocation of the DNS local procedure call 333 leads to in- 
vocation of the user-level DNS service 331. Once the 
communication port is opened, malware 321 may send 
messages to a DNS server, including sending confidential 
information to a malicious DNS server. 
[0068] By monitoring connection requests (e.g., "hooking" the 

"NtCreatePort" API call), the present invention can monitor 
an application's attempt to access a specific service. Typi- 
cally, the vast majority of connection requests will be valid 



and can simply be passed through for processing. Re- 
quests can be intercepted and redirected to the TrueVec- 
tor engine 343 which in turn may determine whether the 
application that originated the request for Internet access 
has appropriate privileges (based on the system's speci- 
fied rules and policy, and/or user interaction). If the appli- 
cation in question has appropriate privileges, its request 
is granted and access to the service is permitted. Other- 
wise, the request is denied, with access to the service be- 
ing blocked. In the latter case, the application is blocked 
from even establishing a communication channel. 
[0069] Another case in which a malicious application (e.g., mal- 
ware 321) may attempt to circumvent security measures is 
to send messages (or other communications) to another 
application (or process). For example, the client-side fire- 
wall manager 341 provides a user interface for receiving 
user input, including providing notification to the user of 
an attempt to access the Internet and receiving user input 
as to whether or not a particular application should be 
permitted to access the Internet. To hide the fact that the 
malware application is attempting to access the Internet, 
the malware application 321 may attempt to shut down or 
disable the firewall manager 341 in one of several ways. 



The malware application may, for example, wait until a 
shared window which is provided across the desktop pops 
up and use this shared window to post user input-based 
messages, such as keystrokes and mouse input, to the 
firewall manager 341. These user input messages may 
seek to block the firewall manager 341 from notifying the 
user of the access attempt or, alternatively, may indicate 
to the firewall manager that access by the malware appli- 
cation should be approved. 
[0070] jo prevent other applications from sending or posting 
certain types of messages (e.g., keyboard messages and 
mouse input messages), the user level components of the 
security system (i.e., firewall manager 341 and TrueVector 
engine 343) are registered with the Interprocess Commu- 
nication Controller (IPCC) subcomponent 349 of the de- 
vice driver 345. These user level components of the secu- 
rity system are registered as a protected application and 
are assigned a protection level that prevents other appli- 
cations from sending or posting certain types of messages 
using these user level components. The types of mes- 
sages that the application wants to be protected against 
(i.e., messages to be blocked) are also specified during 
this registration process. 



[0071] a s a result of the registration of the firewall manager 341, 
when an application (e.g., malware application 321) uses 
the user32 API component 323 to send a message to the 
firewall manager 341 that says "quit", or attempts to give 
specific user interface commands (e.g., a 
"WM.COMMAND" in a Windows environment), the IPCC 
349 intercepts the message and examines it to determine 
whether or not the message is permitted. Typically, the 
IPCC 349 will first look at the source and the destination 
of the message. If the source and the destination are the 
same application or process, then the message is allowed 
to proceed to the destination. However, if the target is a 
protected application or process (e.g., the firewall man- 
ager 341), the IPCC 349 determines whether the message 
is the type of message that should be blocked. In the cur- 
rently preferred embodiment, messages sent to a regis- 
tered application will be blocked (rejected) if they are of 
the type that the registered application is to be protected 
against. The IPCC 349 may include the full decision-mak- 
ing capabilities to make this determination and/or the 
IPCC may consult a rules engine for determining whether 
or not particular types of messages should be blocked. 
For example, the IPCC 349 may block certain communica- 



tions which are prohibited regardless of the circum- 
stances, but may invoke an external rules engine (e.g., a 
rules engine component of TrueVector engine 343) if 
more complex policy arbitration and decision making is 
required. Those skilled in the art will appreciate that there 
are a number of other variations that may be used for de- 
termining whether or not to block particular messages. 
For example, communication privileges may be provided 
only to certain known, approved applications or pro- 
cesses, with other communications being blocked. The 
operations of the IPCC and other components of the secu- 
rity system of the present invention in controlling inter- 
process communications will now be described in greater 
detail. 
Methods of operation 

[0072] pig. 4 comprises a flowchart 400 illustrating the high- 
level methods of operation of the system of the present 
invention in controlling indirect access to networks or re- 
sources by a potentially malicious application. The follow- 
ing description presents method steps that may be imple- 
mented using computer-executable instructions, for di- 
recting operation of a device under processor control. The 
computer-executable instructions may be stored on a 



computer-readable medium, such as CD, DVD, flash 
memory, or the like. The computer-executable instruc- 
tions may also be stored as a set of downloadable com- 
puter-executable instructions, for example, for down- 
loading and installation from an Internet location (e.g., 
Web server). The following discussion uses the operations 
of the system of the present invention in a Microsoft Win- 
dows operating environment as an example, however a 
similar approach may also be used in other operating en- 
vironments. 

[0073] As described above, a particular service (e.g., a DNS ser- 
vice provided by the Windows operating system) may be 
made available to applications through a published port. 
The port, which is an abstraction of a communication 
channel, is associated with a text string that identifies that 
specific port. The operating system kernel typically main- 
tains a list of open ports through which various services 
are made available to calling applications. As shown at 
Fig. 4, the method begins at step 401 with an application 
that may potentially be malicious (e.g., untrusted malware 
application 321 as shown at Fig. 3) requesting a particular 
service (e.g., a DNS service) which is not part of the local 
malware application or process, but rather is a service that 



is handled by another application in a separate process. 
As operating systems (e.g., Windows) are often structured 
to have generic services which perform particular tasks on 
behalf of applications, a particular request by a local ap- 
plication may often lead to the invocation of one or more 
generic services for performing particular tasks. For ex- 
ample, the malware application may use a Windows socket 
function called "gethostbyname" or "getaddrinfo", which 
are Wsock32.dll functions (API calls) that are designed to 
resolve textual representations and obtain an IP address. 
During the process of obtaining the requested IP address, 
a DNS service provided by the operating system may ac- 
cess the Internet to look up an IP address on a remote 
DNS server. 

[0074] As a result of the initial request by the malware applica- 
tion, at step 402 there is an initial invocation of an oper- 
ating system module or function at the "user" privilege 
level (e.g., in the user32 or kernel32 component 323 in 
ring 3 as shown at Fig. 3) for handling of this request in 
the local user process. The user level component deter- 
mines that in order to have the requested service per- 
formed, a communication must be made in a particular 
format on a specific port or communication channel to 



obtain access to the service. The module frequently pro- 
cesses the request (e.g., formatting the request into a 
message in an appropriate format) and may consult a lo- 
cal cache to determine whether the requested information 
is already locally known. For example, the Windows socket 
(e.g., Wsock32.dll) function "gethostbyname", when in- 
voked to resolve a host name to an IP address, will usually 
first check locally for a matching host name entry. If no 
matching host name entry is found locally, this module 
determines that it will then need to invoke a service (e.g., 
a DNS service) to obtain the IP address that matches the 
host name. 

[0075] At step 403, an attempt is made to open a communication 
channel (e.g., "port") to the desired service (e.g., DNS ser- 
vice). To invoke a service, a communication channel must 
first be opened through the operating system kernel (e.g., 
Win32 kernel or NTOS kernel in ring 0 as shown in block 
325 at Fig. 3) to the desired service (e.g., by an operating 
system API call such as "NtCreatePort" for opening a port). 
A communication channel though the kernel is required 
so that the kernel can facilitate communication with the 
requested service or process (e.g., the DNS service) which 
runs in a different address space of the machine. In 



essence, the kernel operating system services are used to 
send messages back and forth to the service or process of 
interest. For example, the kernel32 (at the user level) 
communicates with the NTOS kernel (at the kernel level) 
and indicates that it wants to create a connection to a 
"RPC DNS resolve" service. The NTOS kernel has a list of 
different kinds of services (such as RPC DNS resolve) and a 
list of open ports to such services, which are described by 
a string. The NTOS kernel facilitates communications be- 
tween the application (e.g., malware) and the service (e.g., 
RPC DNS resolve) as these two applications are in different 
memory spaces of the machine. 
[0076] The attempt to open a communication channel (port) to 
the desired service is intercepted and re-routed to the 
IPCC at step 404. The attempt to open a port is a good 
point at which to examine what is occurring as the at- 
tempt provides a good indication of what the application 
(e.g., the malware application in this example) is trying to 
accomplish, depending on the service that is being in- 
voked. The services that are available are typically well- 
known services, so it can be determined what the applica- 
tion is attempting to do through the use of a particular 
service. For example, if the malware application wants to 



communicate with the DNS service, it can be determined 
that the DNS service will attempt a DNS lookup on behalf 
of the malware application (e.g., by looking at parameters 
of the open port request). Additional reasons for inter- 
cepting the attempt to open a port include better error 
handling and reduced impact on system performance 
compared to trapping this activity at a later point in the 
process. Also, it can be more difficult to intercept the ap- 
propriate messages at a later point. 
[0077] | n the currently preferred embodiment, these attempts to 
open a communication channel are intercepted by re- 
routing them from a dispatch table to the IPCC. Commu- 
nications between kernel32 and the NTOS kernel or be- 
tween user32 and the Win32 kernel are based upon a dis- 
patch table. The IPCC generally operates by modifying (or 
patching) this dispatch table to re-route the request for 
particular services to itself so that the IPCC can decide 
whether or not to allow the request before associated 
lower level operating system calls may proceed. At this 
point the IPCC can also determine the currently running 
process or application which is attempting to open the 
connection (e.g., the malware application in this example). 
For example, the "GetCurrentProcess" and "GetCurrentPro- 



cessld" Windows API functions may be invoked to deter- 
mine the current application or process attempting to 
open the connection. 
[0078] At step 405, the IPCC passes the request to the TrueVec- 
tor engine (VS_MON), which includes a rules engine (or 
database), to determine whether or not to block the at- 
tempt by the specified application (e.g., the malware ap- 
plication) to open a channel to the target service (e.g., 
DNS service). In the case of this specific invocation of the 
DNS service, the rules engine of the currently preferred 
embodiment treats this attempt like a request for Internet 
access by the malware application given that ultimately 
the DNS service will obtain Internet access to lookup an IP 
address. The rules database is consulted to determine 
whether the malware application is a known application 
and applies any established rule if the application is 
known. A known application may have an established rule 
which indicates the application is trusted (i.e., approved 
for Internet access), untrusted (blocked from accessing 
the Internet), or which provides for requesting a decision 
from the user (or from a centralized administrative ser- 
vice) about whether or not to permit access. If the appli- 
cation is not known, an established rule may be applied 



(e.g., block Internet access by unknown applications) or 
the user may be informed and asked to decide whether to 
allow access by the malware application using the DNS 
service. 

[0079] As an alternative to consulting an external rules engine, 
the IPCC may include rules and/or logic to evaluate at- 
tempts to access a service itself. In this event, the IPCC 
would examine the attempt and determine whether or not 
to permit access based upon established policies or rules. 
Those skilled in the art will appreciate that a number of 
other variations for evaluating attempts to access a ser- 
vice are also available, which may be appropriately ad- 
dressed by the present invention. For example, the IPCC 
may handle certain access attempts (e.g., deny access to a 
known, "untrusted" application) while invoking an external 
rules engine in other situations, depending on such fac- 
tors as the complexity of the decision-making process 
and the impact of consulting an external rules engine on 
system performance. Further description of a rules engine 
component for security and behavioral policy definition 
and enforcement is provided in commonly-owned Appli- 
cation Serial No. 10/159,820 (Docket No. VIV/0005.01), 
filed May 31, 2002, entitled "System and Method for Secu- 



rity Policy Arbitration," the disclosure of which is hereby 
incorporated by reference in its entirety, including any ap- 
pendices or attachments thereof, for all purposes. 

[0080] At step 406, the rules engine (or the IPCC) determines 
whether to permit or block access to the service by the 
application. In the currently preferred embodiment, the 
rules engine of the TrueVector engine returns a status of 
"yes" (allow access), "no" (deny or block access), or "ask" 
(ask for permission). "Ask" may include asking the user or 
a non-user such as a centralized administrative service 
about whether or not to permit access by the application. 
If an external rules engine was consulted, the answer is 
returned to the IPCC by the rules engine. 

[0081] At step 407, the IPCC permits or blocks the attempt by the 
application to access the service. If the answer that is de- 
termined at step 406 is to allow the request for service, 
the request is forwarded to the original destination rou- 
tine (e.g., by consulting a fix-up table that is maintained 
to track the handling of the re-routed requests). In other 
words, the communication (or request) that was re-routed 
to the IPCC is re-dispatched to the originally specified re- 
cipient (e.g., the DNS service). Otherwise, if the answer is 
to block (deny) the attempt, then a response is returned to 



the requesting application (e.g., malware) with an appro- 
priate error message, such as "connection refused", "ser- 
vice not available", or the like. After the attempt is allowed 
or denied, the handling of this exemplary request is com- 
pleted. At this point the system reenters a steady state 
ready to handle future requests for access. 
[0082] Another aspect of the present invention is to provide a 
mechanism for regulating interprocess communication 
amongst applications and services so as to prevent a 
rogue application from controlling or misusing another 
application or service. Applications (or services) to be pro- 
tected are registered with the security system of the 
present invention and aspects regarding how these appli- 
cations are to be protected are described. Communica- 
tions with registered applications are then regulated by 
the security system of the present invention as hereinafter 
described. 

[0083] pig. 5 comprises a flowchart 500 illustrating the high- 
level methods of operation of the system of the present 
invention in regulating interprocess communications be- 
tween applications. As with the method in Fig. 4, the fol- 
lowing description presents method steps that may be 
implemented using computer-executable instructions, for 



directing operation of a device under processor control. 
The computer-executable instructions may be stored on a 
computer-readable medium, such as CD, DVD, flash 
memory, or the like. The computer-executable instruc- 
tions may also be stored as a set of downloadable com- 
puter-executable instructions, for example, for down- 
loading and installation from an Internet location (e.g., 
Web server). The following discussion uses the operations 
of the system of the present invention in a Microsoft Win- 
dows operating environment as an example, however a 
similar approach may also be used in other operating en- 
vironments. 

[0084] The method commences at step 501 with the registration 
of a particular application (e.g., the ZoneAlarm firewall 
manager 341 as shown at Fig. 3). In the presently pre- 
ferred embodiment, an application to be protected is reg- 
istered with the Interprocess Communication Controller 
(IPCC) component 349 of the TrueVector device driver 345 
through the use of the TrueVector engine (VS_MON) 343, 
all as shown at Fig. 3. As previously described, the firewall 
manager application can be registered as a protected ap- 
plication and assigned a protection level that prevents 
other applications from posting particular types of mes- 



sages (communications) to the firewall manager. The 
types of messages that the application is to be guarded 
against (i.e., messages to be blocked) are also specified 
during this registration process. 
[0085] At step 502, a potentially malicious application (e.g., mal- 
ware application 321 as shown at Fig. 3), attempts to send 
a message to the registered application (e.g., the 
ZoneAlarm firewall manager 341 as shown at Fig. 3). The 
malware application may send a wide variety of communi- 
cations (messages) to the firewall manager in an attempt 
to disable the firewall manager or otherwise circumvent 
the security measures provided by the security system. 
For example, the firewall manager may, from time to time, 
present a dialog box in the user interface to notify the 
user of particular events. This dialog box may, for in- 
stance, pop-up to inform the user of an attempt to access 
the Internet by a given application and request a decision 
from the user as to whether or not such access should be 
permitted. The malware application may attempt to dis- 
able or circumvent the firewall (end point security module) 
by waiting until this dialog box is presented, and then 
sending user input-based messages, such as keystrokes 
and mouse input, to the firewall manager. For example, 



input may be sent which indicates that the malware appli- 
cation should be permitted to access the Internet and/or 
that it should be categorized as a trusted application (i.e., 
added to a list of applications permitted to access the In- 
ternet). 

[0086] There are a number of other ways by which a malicious 
application may try to attack another application (e.g., an 
end point security application). The malware application 
may send user input messages using a Microsoft Windows 
"SendMessage" function (or "PostMessage", or the like) 
which sends a message to one or more specified win- 
dow^) as described above. As another example, the mal- 
ware application could also send another application a 
message to quit (i.e., cease operating) or send other mes- 
sages (e.g., "WM.COMMAND" messages) seeking to ex- 
ploit vulnerabilities in the application and/or operating 
system to breach security. Those skilled in the art will ap- 
preciate that there are a number of ways in which a mali- 
cious application can attempt to disable or circumvent an- 
other application. As another example, in the case of Mi- 
crosoft Outlook, the Outlook application publishes certain 
communication channels that are not Windows messages, 
but may comprise ports or other ways of communicating. 



These communication channels are capable of being 
abused by rogue applications. One of the difficulties in 
regulating abusive messages is that it is often difficult to 
determine the source of a given message. An application 
at the user level of the system (such as the firewall man- 
ager in this example) may be unable to defend itself 
against this type of attack because it cannot determine 
whether or not a given request that is received is legiti- 
mate. 

[0087] At step 503, the message sent by the application is inter- 
cepted at the kernel level and redirected (or re-routed) to 
the IPCC. A message sent via the "SendMessage" function 
at the user level is normally received and translated at the 
kernel level before being dispatched to its intended desti- 
nation, and a dispatch table is typically employed at the 
kernel level similar to that previously described above. In 
the currently preferred embodiment, the message (or 
communication) is intercepted by modifying this dispatch 
table to re-route messages to the IPCC so that the IPCC 
can decide whether or not to permit the communication to 
be sent to the target destination. 

[0088] Next, at step 504 information about the message, the 
source application (process), and the target application 



are collected. This information is typically extracted from 
the communication (e.g., extracted from Windows mes- 
sage parameters). The specific information that is ex- 
tracted or otherwise obtained by the IPCC component of 
the TrueVector device driver varies depending upon the 
type of message or communication that is involved. 
[0089] Based upon the collected information about the message, 
the source application, and the target, at step 505 a de- 
termination is made as to whether the communication 
should be permitted (i.e., allowed to proceed to the tar- 
get) or blocked. This determination typically first involves 
determining whether the target application is a protected 
application (i.e., the target application was previously reg- 
istered as a protected application as described above). If 
the target application is a protected application, the type 
of message (communication) is then evaluated to deter- 
mine if it is a type of communication that the target appli- 
cation is to be protected against (based upon the rules 
established above regarding what is protected). In the 
currently preferred embodiment, this determination is 
made directly in the TrueVector device driver at the kernel 
level (i.e., without the need to consult an external rules 
engine) in the interests of minimizing the impact on sys- 



tern performance. However, the determination may also 
be made by an external rules engine, as desired. 

[0090] As described above with respect to evaluation of attempts 
to access a communication channel, those skilled in the 
art will appreciate that a number of alternatives are avail- 
able about how the determination to allow or reject com- 
munications may be made. For example, the device driver 
(IPCC) may make the determination itself in many cases 
(e.g., for particular types of messages) and may invoke an 
external rules engine to handle other types of communi- 
cations. As another example, rules may be established to 
provide that the device driver should reject all messages 
to a first target application, should allow all messages to a 
second target application, and should refer all messages 
to a third target to an external rules engine (e.g., a rules 
engine component of the TrueVector engine), and so on 
and so forth. In some cases the rules engine may also, in 
turn, consult additional resources (e.g., an external web 
server on a different machine or location) in the process 
of determining whether or not to permit a given commu- 
nication attempt. 

[0091] Based upon the determination made by the device driver 
and/or rules engine, at step 506 the message (or commu- 



nication) is either permitted (i.e., allowed to proceed) or 
rejected. If the message is allowed to proceed, the mes- 
sage is re-dispatched to the original target (e.g., by con- 
sulting a fix-up table that is maintained to track the han- 
dling of messages). Otherwise, if the decision is made to 
block the communication, then a response is returned to 
the requesting application (e.g., malware) with an appro- 
priate error message. After the attempt is allowed or de- 
nied, the handling of this message is completed. The 
above process may then be repeated for regulating addi- 
tional interprocess communications. 
Detailed internal operation 

[0092] Components of Interprocess Communication Controller 

[0093] pig. 6 is a block diagram illustrating the components of 
the Interprocess Communication Controller (IPCC) 349 of 
the currently preferred embodiment. As shown, the com- 
ponents include a patcher module 610, a protected pro- 
cess registry 620, an external communication facility 625, 
handlers 630, and a disposition module 640. Each of 
these components will now be described. 

[0094] The patcher module 610 serves to intercept and redirect 
operating system calls to handlers 630 of the IPCC 349. 



More particularly, the patcher module 610 locates the dis- 
patch tables that are used by the system to take a call re- 
ceived from the user level and dispatch the call to the ap- 
propriate lower level operating system routines of the 
kernel as illustrated at Fig. 3. The patcher module 610 
generally redirects the call to the IPCC by replacing the 
original destination in the dispatch table with a pointer to 
the appropriate handler routines 630 of the IPCC. The 
original destination address is also retained as hereinafter 
described. In this fashion operating system calls are redi- 
rected to the handlers 630 for determining whether or not 
the communication (call) should be allowed or blocked. 
The original destination information from the dispatch ta- 
ble is retained so that the call can be forwarded to the 
original destination if the IPCC authorizes (i.e., permits) 
the communication. 
[0095] The protected process registry 620 provides a facility for 
registering a process (or other protected items) in order to 
protect such process (or item) from other processes as 
provided in one embodiment of the present invention. A 
user or administrator may register a process that is to be 
protected with the protected process registry 620 of the 
IPCC, so that the IPCC can determine whether or not to 



permit other processes or applications to communicate 
with the registered process. The protected process reg- 
istry 620 may also (optionally) specify what types of pro- 
tection should be provided (e.g., specifying communica- 
tions a process should be protected against). 

[0096] The external communication facility 625 provides for 

(optional) forwarding of certain items to an external ser- 
vice (e.g., the rules engine of the TrueVector engine). In 
the currently preferred embodiment, instead of making all 
decisions locally within the IPCC, in certain cases a mes- 
sage may be sent to an external service (e.g., the rules 
engine of the TrueVector engine) asking the external ser- 
vice to make a decision as to whether to permit or block a 
particular communication. In addition, the external com- 
munication facility 625 may also send messages syn- 
chronously or asynchronously to an external service (e.g., 
the TrueVector engine) to inform an external service of 
specific activity and/or to log certain actions that have 
been taken by the IPCC. 

[0097] The handlers 630 are the actual handlers that examine 
and make decisions regarding intercepted communica- 
tions or calls. The handlers 630 look at calls (i.e., mes- 
sages or communications) that have been redirected to 



the IPCC and their parameters and make decisions about 
how to handle them. For example, a handler may decide 
whether a call can be handled locally or should be for- 
warded to another module (e.g., the TrueVector engine) 
for decision making. A handler may also decide whether a 
call or message can be forwarded on to its original desti- 
nation or blocked. 

[0098] The disposition module 640 manages appropriate routing 
of a communication after its disposition has been deter- 
mined. A handler that is handling a communication will 
usually have a disposition for the communication (i.e., to 
allow or to block the communication). If a call is allowed, 
the disposition module 640 causes the call to be for- 
warded on to its original destination. For example, if a 
DNS service was originally invoked, the disposition mod- 
ule 640 would cause the call to be passed on to the DNS 
service if the handler 630 determined that the call was al- 
lowable. However, if the handler determined that a call or 
message is to be blocked, an error message or other ap- 
propriate indication is returned to the calling process or 
application to indicate this fact. 

[0099] intercepting and monitoring local procedure calls 

[0100] The following "vslpc.nt.c" module is a component of the 



TrueVector driver that intercepts and monitors local pro- 
cedure calls (LPCs) in a Windows NT environment. 
[0101] i; /* vslpc.nt.c (vsdatant.sys) 

2: Driver component for WinNT, hooking/monitoring of LP 
C calls */ 

3: #include "ntddk.h" 
4: #include "stdarg.h" 
5: #include "stdio.h" 
6: #include "vsdatant.h" 
7: #include "vserror.h" 
8: #include "vsdriver.h" 
9: #ifdef .DEBUG 
10: #define _DEBUG_LPC 
11: #endif 

12: // Various data structures 

13: typedef struct _LPC_SECTION_OWNER_M EMORY { 

14: ULONG Length; 

15: HANDLE SectionHandle; 

16: ULONG OffsetlnSection; 

17: ULONG ViewSize; 

18: PVOID ViewBase; 

19: PVOID OtherSideViewBase; 

20: } LPC_SECTION_OWNER_MEMORY, *PLPC_SECTION_OW 



NER.MEMORY; 



21 
22 
23 
24 
25 
26 
27 
28 
29 
30 
31 
32 
33 
34 
35 
36 
37 
38 



typedef struct _LPC_SECTION_M EMORY { 

ULONG Length; 

ULONG ViewSize; 

PVOID ViewBase; 
} LPC_SECTION_MEMORY, *PLPC_SECTION_MEMORY; 
typedef struct _LPC_MESSAGE { 

USHORT DataLength; 

USHORT Length; 

USHORT MessageType; 

USHORT DatalnfoOffset; 

CLIENTJD Clientld; 

ULONG Messageld; 

ULONG Callbackld; 
} LPC.MESSAGE, *PLPC_MESSAGE; 
// LPC call prototypes 
typedef NTSTATUS 
(NTAPI 

*NT_CONNECT_PORT) ( OUT PHANDLE ClientPo 



rtHandle, 



39 
40 
41 



IN PUNICODE_STRING ServerPortName, 

IN PSECURITY_QUALITY_OF_SERVICE SecurityQos, 

IN OUT PLPC_SECTION_OWNER_M EMORY ClientSha 



redMemory, 

42: OUT PLPC_SECTION_MEMORY ServerSharedMemor 

y, 

43: OUT PULONG MaximumMessageLength, 

44: IN OUT PVOID Connectionlnfo, 

45: IN OUT PULONG ConnectionlnfoLength); 

46: NTSYSAPI 

47: NTSTATUS 

48: NTAPI 

49: NtConnectPort( OUT PHANDLE ClientPortHan 
die, 

50: IN PUNICODE.STRING ServerPortName, 

51: IN PSECURITY_QUALITY_OF_SERVICE SecurityQos, 

52: IN OUT PLPC_SECTION_OWNER_M EMORY ClientSha 

redMemory, 

53: OUT PLPC_SECTION_MEMORY ServerSharedMemor 

y, 

54: OUT PULONG MaximumMessageLength, 

55: IN OUT PVOID Connectionlnfo, 

56: IN OUT PULONG ConnectionlnfoLength); 
57: #ifdef _DEBUG_LPC 

58: #define SECURE_CONNECT_PORT_SERVICE_NT 0x00 
000000 



59: #define SECURE_C0NNECT_P0RT_SERVICE_2K 0x00 
0000b8 

60: #define SECURE_CONNECT_PORT_SERVICE_XP 0x00 
0000d2 

61: typedef NTSTATUS 
62: (NTAPI 

63: *NT_SECURE_CONNECT_PORT) (OUT PHANDLE Clien 
tPortHandle, 

64: IN PUNICODE_STRING ServerPortName, 

65: IN PSECURITY_QUALITY_OF_SERVICE SecurityQos, 

66: IN OUT PLPC_SECTION_OWNER_M EMORY ClientSha 

redMemory, 

67: PVOID pUnknown, 

68: OUT PLPC_SECTION_M EMORY ServerSharedMemor 

y, 

69: OUT PULONG MaximumMessageLength, 

70: IN OUT PVOID Connectionlnfo, 

71: IN OUT PULONG ConnectionlnfoLength); 

72: /* Not exported by ntoskrnl 

73: NTSYSAPI 

74: NTSTATUS 

75: NTAPI 

76: NtSecureConnectPort(OUT PHANDLE ClientPortH 



andle, 

77: IN PUNICODE_STRING ServerPortName, 

78: IN PSECURITY_QUALITY_OF_SERVICE SecurityQos, 

79: IN OUT PLPC_SECTION_OWNER_M EMORY ClientSha 

redMemory, 

80: PVOID pUnknown, 

81: OUT PLPC_SECTION_MEMORY ServerSharedMemor 

y, 

82: OUT PULONG MaximumMessageLength, 

83: IN OUT PVOID Connectionlnfo, 

84: IN OUT PULONG ConnectionlnfoLength); 

85: */ 

86: #define CREATE_PORT_SERVICE_NT 0x00000000 
87: #define CREATE_PORT_SERVICE_2K 0x00000028 
88: #define CREATE_PORT_SERVICE_XP 0x0000002e 
89: typedef NTSTATUS 
90: (NTAPI 

91: *NT_CREATE_PORT) (OUT PHANDLE PortHandle, 

92: IN POBJECT_ATTRIBUTES ObjectAttributes, 

93: IN ULONG MaxConnectlnfoLength, 

94: IN ULONG MaxDataLength, 

95: IN OUT PULONG Reserved OPTIONAL ); 

96: /* Not exported by ntoskrnl 



97: NTSYSAPI 
98: NTSTATUS 
99: NTAPI 

100: NtCreatePort(OUT PHANDLE PortHandle, 
101: IN POBJ ECT_ATTRI BUTES ObjectAttributes, 
102: IN ULONG MaxConnectlnfoLength, 
103: IN ULONG MaxDataLength, 
104: IN OUT PULONG Reserved OPTIONAL ); 

105: */ 

106: #endif //_DEGUG_LPC 

107: // Hook handlers and handles 

108: NT.CONNECT.PORTConnectPortHandler = NULL; 

109: HOOK.FUNCTION hConnectPort = {0}; 

110: #ifdef _DEBUG_LPC 

111: NT_SECURE_CONNECT_PORT SecureConnectPortHand 
ler = NULL; 

112: HOOK.FUNCTION hSecureConnectPort = {0}; 
113: NT_CREATE_PORT CreatePortHandler = NULL; 
114: HOOK.FUNCTION hCreatePort = {0}; 
115: #endif //_DEBUG_LPC 

116: NTSTATUS __stdcall OnProcessLpcDnsAccess(DWORD 
dwProcessID) 

117: { // We are generating a "pseudo" WSock message h 



ere, not 

a process message 

118: PVSMSG.STREAM pMsg; 

119: PHOOKREQUEST pHook = pWSockHook; 

120: NTSTATUS Status = STATU S.SUCC ESS; 

121: if (dwProcessID == dwMonitorProcessID) 

122: return STATUS.SUCCESS; 

123: if(pHook) 

124: { pMsg = NewMessage(pHook, sizeof(VSMSG_STRE 
AM), 

GetCurrentProcesslDO, 

125: GetCurrentThreadlDO, 0, 0,0); 

126: if(pMsg) 

127: { pMsg->dwMsgFlags |= MFM.NEEDREPLY; 
128: pMsg->dwMsgl_evel = MSG_LEVEL_INFO_LOW; 
129: Status = PutMessage(pHook, &pMsg, 
MCWSOCK_LPC_DNS_ACCESS_BEFORE, 
130: NULL, 0, 0); 

131: FreeMessage(pHook, (PBASEVSMSG)pMsg); 
132: if (Status) 

133: Status = STATUS_ACCESS_DENIED; 
134: } 
135: } 



136: return Status; 
137: } 

138: NTSTATUS 
139: NTAPI 

140: HookConnectPort( OUT PHANDLE ClientPort 
Handle, 

141: IN PUNICODE_STRING ServerPortName, 
142: IN PSECURITY_QUALITY_OF_SERVICE SecurityQos, 
143: IN OUT PLPC_SECTION_OWNER_MEMORY ClientSh 
ared Memory, 

144: OUT PLPC_SECTION_MEMORY ServerSharedMemo 

ry, 

145: OUT PULONG MaximumMessageLength, 

146: IN OUT PVOID Connectionlnfo, 

147: IN OUT PULONG ConnectionlnfoLength) 

148: { NTSTATUS Status = STATUS.SUCCESS; 

149: CHAR cPortName[256] = ""; 

150: DWORD dwLen = 0; 

151: DWORD dwProcessID = GetCurrentProcesslDO; 
152: if (ServerPortName) 

153: dwLen = UnicodeStringToChar(*ServerPortName, c 

PortName, 

sizeof(cPortName)); 



154: // DNS request to services.exe 

155: if (lstrcmpi(cPortName, "\\RPC Control\\DNSResolv 

er") == 0) 

156: { Status = OnProcessLpcDnsAccess(dwProcesslD); 
157: } 

158: if ((Status == STATUS.SUCCESS)) 
159: { if (ConnectPortHandler) 

160: { Status = ConnectPortHandler(ClientPortHandle, 
ServerPortName, 

161: SecurityQos, 

ClientSharedMemory, ServerSharedMemory, 

162: MaximumMessageLength, Connectionlnfo, 

163: ConnectionlnfoLength); 

164: } 

165: else 

166: { Status = NtConnectPort( ClientPortHandle, 
ServerPortName, SecurityQos, 

167: ClientSharedMemory, ServerSharedMemory, 

MaximumMessageLength, Connectionlnfo, 
168: ConnectionlnfoLength); 
169: } 
170: } 

171: #ifdef .DEBUG 



172: if (Status == STATUS.SUCCESS) 

173: DbgPrint("[LPC NtConnectPort] m, %s"" (%x) OK - Por 

t%x\n", 

174: cPortName, dwProcessID, ClientPortHandle ? 

*ClientPortHandle : 0); 
175: else 

176: DbgPrint("[LPC NtConnectPort] m, %s"" (%x) FAIL - St 
atus %x\n", 

177: cPortName, dwProcessID, Status); 
178: #endif //.DEBUG 
179: return Status; 
180: } 

181: #ifdef _DEBUG_LPC 
182: NTSTATUS 
183: NTAPI 

184: HookSecureConnectPort( OUT PHANDLE Clien 
tPortHandle, 

185: IN PUNICODE_STRING ServerPortName, 
186: IN PSECURITY_QUALITY_OF_SERVICE SecurityQos, 
187: IN OUT PLPC_SECTION_OWNER_MEMORY ClientSh 
aredMemory, 

188: PVOID pUnknown, 

189: OUT PLPC_SECTION_MEMORY ServerSharedMemo 



ry, 

190: OUT PULONG MaximumMessageLength, 

191: INOUTPVOID Connectionlnfo, 

192: IN OUT PULONG ConnectionlnfoLength) 

193: { NTSTATUS Status = STATUS.SUCCESS; 

194: CHAR cPortName[256] = ""; 

195: DWORD dwLen = 0; 

196: DWORD dwProcessID = GetCurrentProcesslDO; 
197: if (ServerPortName) 

198: dwLen = UnicodeStringToChar(*ServerPortName, c 

PortName, 

sizeof(cPortName)); 

199: // DNS request to services.exe 

200: if (IstrcmpKcPortName, "\\RPC Control\\DNSResolv 

er") ==0) 

201: { Status = OnProcessLpcDnsAccess(dwProcesslD); 
202: } 

203: if ((Status == STATUS.SUCCESS)) 
204: { if (ConnectPortHandler) 

205: { Status = SecureConnectPortHandler( ClientPort 
Handle, 

ServerPortName, 

206: SecurityQos, ClientSharedMemory, pUnknown, 



ServerShared Memory, 

207: MaximumMessageLength, Connectionlnfo, 

ConnectionlnfoLength); 

208: } 

209: else 

210: { 

211: /* Status = NtSecureConnectPort( ClientPortHandl 
e, 

ServerPortName, SecurityQos, 

212: ClientSharedMemory, pUnknown, ServerShared 

Memory, 

MaximumMessageLength, 

213: Connectionlnfo, ConnectionlnfoLength);*/ 

214: Status = STATUS_ACCESS_DENIED; 
215: } 
216: } 

217: #ifdef .DEBUG 

218: if (Status == STATUS.SUCCESS) 

2 19: DbgPrint("[LPC NtSecureConnectPort] ""%s"" (%x) O 

K- Port%x\n", 

220: cPortName, dwProcessID, ClientPortHandle ? 

*ClientPortHandle : 0); 
221: else 



222: DbgPrint("[LPC NtSecureConnectPort] m, %s"" (%x) F 

AIL - Status 

%x\n", 

223: cPortName, dwProcessID, Status); 
224: #endif //.DEBUG 
225: return Status; 
226: } 

227: NTSTATUS 
228: NTAPI 

229: HookCreatePort( OUT PHANDLE PortHandle, 
230: IN POBJECT_ATTRIBUTES ObjectAttributes, 
231: INULONG MaxConnectlnfoLength, 
232: IN ULONG MaxDataLength, 
233: IN OUT PULONG Reserved OPTIONAL ) 

234: { NTSTATUS Status = STATU S.SUCC ESS; 
235: CHAR cPortName[256] = ""; 
236: DWORD dwLen = 0; 

237: DWORD dwProcessID = GetCurrentProcesslDO; 
238: if (ObjectAttributes && ObjectAttributes->ObjectNa 
me) 

239: dwLen = UnicodeStringToChar(*(ObjectAttributes- 
>ObjectName), 

cPortName, sizeof(cPortName)); 



240: if ((Status == STATUS.SUCCESS)) 
241: { if (CreatePortHandler) 

242: { Status = CreatePortHandler( PortHandle, ObjectA 
ttributes, 

MaxConnectlnfoLength, 

243: MaxDataLength, Reserved); 

244: } 

245: else 

246: { /*Status = NtCreatePort( PortHandle, ObjectAttri 
butes, 

MaxConnectlnfoLength, 

247: MaxDataLength, Reserved); */ 

248: Status = STATUS_ACCESS_DENIED; 

249: } 
250: } 

251: #ifdef .DEBUG 

252: if (Status == STATUS.SUCCESS) 

253: DbgPrint("[LPC NtCreatePort] m, %s"" (%x) OK - Port 

%x\n", 

254: cPortName, dwProcessID, PortHandle ? *PortH 

andle : 0); 
255: else 

256: DbgPrint("[LPC NtCreatePort] m, %s ,m (%x) FAIL - Stat 



us %x\n", 

257: cPortName, dwProcessID, Status); 
258: #endif //.DEBUG 
259: return Status; 
260: } 

261: #endif //_DEGUG_LPC 
262: 

263: NTSTATUS StartTrackLPCO 
264: { 

265: NTSTATUS Status; 

266: Status = Hooklnt2EService( &hConnectPort, HookCo 
nnectPort, 

267: Findlnt2EService(NtConnectPort, 0)); 

268: if (Status == STATUS.SUCCESS) 

269: ConnectPortHandler = hConnectPort.pOldFunction 

270: else 

271: Status = STATUS.UNSUCCESSFUL; 
272: #ifdef _DEBUG_LPC 

273: Status = Hooklnt2EService( &hSecureConnectPort, 
274: HookSecureConnectPort, 

275: Findlnt2EServiceBylD(SECURE_CONNECT_PORT_S 
ERVICE)); 



276: if (Status == STATUS.SUCCESS) 

277: SecureConnectPortHandler = hSecureConnectPort. 

pOldFunction; 

278: else 

279: Status = STATUS.UNSUCCESSFUL; 

280: Status = Hooklnt2EService( &hCreatePort, HookCrea 

tePort, 

281: Findlnt2EServiceBylD(CREATE_PORT_SERVICE)); 
282: if (Status == STATUS.SUCCESS) 
283: CreatePortHandler = hCreatePort.pOldFunction; 
284: else 

285: Status = STATUS.UNSUCCESSFUL; 
286: #endif //_DEGUG_LPC 
287: return Status; 
288: } 

289: NTSTATUS StopTrackLPCO 
290: { 

291: NTSTATUS Status; 

292: Status = Unhooklnt2EService(&hConnectPort); 
293: #ifdef _DEBUG_LPC 

294: Status = Unhooklnt2EService(&hSecureConnectPort) 
295: Status = Unhooklnt2EService(&hCreatePort); 



296: #endif //_DEGUG_LPC 
297: return Status; 
298: } 

[0102] a "StartTrackLPCO" function commencing at line 263 of 
the above "vslpc.nt.c" module is an initialization routine. 
As shown at lines 266-267 a call is made to a 
"Hooklnt2EService" utility routine that asks it to hook (i.e., 
intercept calls to) a particular routine. In this case, the 
utility routine is called to hook a routine named "NtCon- 
nectPort" which is a routine that a process or application 
would use to attempt to communicate with a service such 
as the DNS service. The "Hooklnt2EService" utility routine 
locates the correct dispatch table and obtains an index 
into the dispatch table ("Findlnt2EService"). The "Start- 
TrackLPCO" function also indicates the address to which 
the call should be redirected, and returns a data structure 
that includes the address to which the call was originally 
directed. The data structure is retained so that the ad- 
dress to which the call was initially directed can be re- 
trieved. In this example, two other functions that are of 
interest are also hooked in the same manner. A "SE- 
CURE_CONNECT_PORT_SERVICE" is hooked as provided at 
lines 273-275. In addition, calls to a "CRE- 



ATE_PORT_SERVICE" function are intercepted as shown at 
lines 280-281. 

[0103] The "StopTrackLPCO" routine which commences above at 
line 289 is basically the inverse of the "StartTrackLPCO" 
function. The "StopTrackLPCO" routine "unhooks" the 
specified functions and restores the dispatch table to its 
original state. 

[0104] The "HookConnectPort" routine at lines 140-180 is a rou- 
tine that is called by the above-described "Start- 
TrackLPCO" function. Of particular interest, it converts a 
"ServerPortName" parameter to a character string as 
shown at line 153. The resulting character string is then 
compared to the known string name of a service "\\RPC 
Control\\DNSResolver" as shown at line 155. In this ex- 
ample, the known string name represents the known 
name of the DNS service which is the name used system- 
wide to identify the DNS service. If the comparison is pos- 
itive, a utility routine named "OnProcessLpcDnsAccess" is 
then called as shown at line 156. This utility routine sends 
a message to the user level and invokes the user level to 
determine whether or not to permit DNS access. 

[0105] The "OnProcessLpcDnsAccess" function (described in 

more detail below) may cause a pop-up to be displayed in 



the user interface to ask the user as to whether or not to 
permit access to the DNS service. Alternatively, an estab- 
lished policy or rule may be consulted to determine 
whether or not access should be permitted. As also shown 
at line 156, an identifier of the calling process 
("dwProcessID") is used to identify the calling process that 
is requesting access to the DNS service. A decision may 
then be made by consulting the user or applying an es- 
tablished policy as to whether this particular calling pro- 
cess should be permitted to use the DNS service. 

[0106] |f a status of "STATUS.SUCCESS" is returned as provided at 
line 158 (and the original handler was saved as provided 
at line 159), the parameters of the original call are passed 
on to the appropriate operating system routine as shown 
at lines 160-164. If a status is other than "STA- 
TUS_SUCCESS" is returned, the status is simply returned as 
provided at line 179. For example, a status of "access de- 
nied" or "access failed" may be returned to the calling 
process or application in the event that the calling appli- 
cation is not permitted to access the DNS service. It 
should be noted that the status information that is re- 
turned at line 179 may indicate either success or failure. 

[0107] The "HookSecureConnectPort" routine illustrated at lines 



184-226 and the "HookCreatePort" routine illustrated at 
lines 229-260 operate in essentially the same manner as 
the "HookConnectPort" routine described above. These 
other routines may be called by the "StartTrackLPCO" 
function depending on the service that the calling applica- 
tion is attempting to access. Calls to these routines may 
then be evaluated in the same manner as previously de- 
scribed. 

[0108] The "OnProcessLpcDnsAccess" routine is illustrated at 
lines 116-137. As described above and as shown at line 
116, the ID of the calling process is received as an input 
parameter to this routine. The process ID uniquely identi- 
fies the process or application that is attempting to access 
the DNS service. A message ("pMsg") is then created in an 
internal messaging format ("NewMessage") and two flags 
are set as shown at lines 124-128. The first flag is 
"MFM.NEEDREPLY" which indicates that the underlying 
messaging transports should wait for a reply before the 
call to the DNS service is allowed to proceed. Next at lines 
129-130 the "PutMessage" is a pseudo Windows socket 
message that is called before execution. The result that is 
returned is a status indicating whether or not the commu- 
nication (i.e., access to the DNS service) is to be permit- 



ted. Next, at lines 131-133 the message structure is freed 
and if any status is returned (i.e., "if (Status)"), the status 
is changed to "STATUS_ACCESS_DENIED" which is a stan- 
dard error message that is understood by the calling 
function. The status is then returned as provided at line 
136. The second aspect of the present invention in which 
a registered application is protected for certain communi- 
cations will next be described. 
[0 1 09] interception and monitoring of GDI calls 

[0110] The following "vsgdLnt.c" module is a component of the 
TrueVector driver that intercepts and monitors GDI 
(graphic device interface) messages in a Windows NT en- 
vironment: 

[0111] i: /* vsgdLnt.c (vsdatant.sys) 

2: Driver component for WinNT, hooking/monitoring of 

GDI calls 

3:*/ 

4: #include "ntddk.h" 
5: #include "stdarg.h" 
6: #include "stdio.h" 
7: #include "vsdatant.h" 
8: #include "vserror.h" 
9: #include "vsdriver.h" 



10: #define WM.KEYFIRST 

11: #define WM.KEYDOWN 

12: #define WM.KEYUP 

13: #define WM.CHAR 

14: #define WM.DEADCHAR 

15: #define WM.SYSKEYDOWN 

16: #define WM.SYSKEYUP 

17: #define WM.SYSCHAR 

18: #define WM.SYSDEADCHAR 

19: #define WM.KEYLAST 

20: #define WM.COMMAND 

21: #define WM_SYSCOMMAND 

22: #define WM.TIMER 

23: #define WM.MOUSEFIRST 

24: #define WM.MOUSEMOVE 

25: #define WM.LBUTTONDOWN 

26: #define WM.LBUTTONUP 

27: #define WM.LBUTTONDBLCLK 

28: #define WM.RBUTTONDOWN 

29: #define WM.RBUTTONUP 

30: #define WM.RBUTTONDBLCLK 

31: #define WM_M BUTTON DOWN 

32: #define WM.MBUTTONUP 



0x0100 

0x0100 
0x0101 
0x0102 
0x0103 
0x0104 
0x0105 
0x0106 

0x0107 
0x0108 
0x0111 
0x0112 
0x0113 
0x0200 
0x0200 

0x0201 
0x0202 
0x0203 
0x0204 
0x0205 
0x0206 
0x0207 
0x0208 



33: 

34: 

35: 

36: 

37: 

38: 

39: 

40: 

41: 

42: 

43: 

44: 

45: 

46: 

47: 

00 

48: 

be 

49: 

cc 

50: 

51: 

52: 



#defi 
#defi 
#defi 
#defi 
#defi 
#defi 
#defi 
#defi 
#defi 
#defi 
#defi 
#defi 
#defi 



ne WM.MBUTTONDBLCLK 
ne WM.MOUSEWHEEL 
ne WM_XBUTTONDOWN 
ne WM_XBUTTONUP 
ne WM_XBUTTONDBLCLK 
ne WM.MOUSELAST 
ne BM_S ETC HECK 
ne BM.GETSTATE 
ne BM.SETSTATE 
ne BM.SETSTYLE 
ne BM.CLICK 
ne BM.GETIMACE 
ne BM.SETIMAGE 



0x0209 
0x020A 

0x020B 
0x020C 

0x020D 
0x020D 



OxOOFl 
0x00F2 
0x00F3 
0x00F4 
0x00F5 

0x00F6 
0x00F7 
// GDI call prototypes 

#define USER_MESSAGE_CALL_SERVICE_NT 0x000000 

#define USER_MESSAGE_CALL_SERVICE_2K 0x000011 

#define USER_MESSAGE_CALL_SERVICE_XP 0x000011 

typedef NTSTATUS 
(NTAPI 

* NT_ U S E R_ M ESS AG E_C ALL) ( HWND hWnd, 



53: UINT Msg, WPARAM wParam, 

54: LPARAM IParam, LRESULT IResult, 

55: DWORD dwUnknownl, DWORD dwUnknown2); 

56: #define USER_POST_MESSAGE_SERVICE_NT 0x000000 

00 

57:#define USER_POST_MESSAGE_SERVICE_2K 0x000011 
cb 

58:#define USER_POST_MESSAGE_SERVICE_XP 0x000011 
db 

59: typedef NTSTATUS 
60: (NTAPI 

61: *NT_USER_POST_MESSAGE) ( HWND hWnd, 
62: UINT Msg, WPARAM wParam, 
63: LPARAM IParam); 

64: #define USER_SEND_INPUT_SERVICE_NT 0x0000000 
0 

65:#define USER_SEND_INPUT_SERVICE_2K OxOOOOlle 
1 

66:#define USER_SEND_INPUT_SERVICE_XP 0x000011f6 
67: typedef UINT 
68: (NTAPI 

69: *NT_USER_SEND_INPUT) ( UINT nlnputs, // count of 
input events 



70: PVOID plnput, // struct _LPINPUT plnputs - arr 
ay of 

input events 

71: int cbSize // size of structure 

72:); 

73:#define USER_QUERY_WINDOW_SERVICE_NT 0x0000 
0000 

74: #define USER_QUERY_WINDOW_SERVICE_2K 0x0000 
lld2 

75:#define USER_QUERY_WINDOW_SERVICE_XP 0x0000 
lle3 

76: #define QUERY_WINDOW_PROCESS 0x00000000 
77: #define QU E RY_WI N DOW_TH R EAD 0x00000001 
78: typedef DWORD 
79: (NTAPI 

80: *NT_USER_QUERY_WINDOW) ( HWND hWnd, DWORD d 

wQuery); // See 

QUERY_WINDOW_??? 

81: #define USER_GET_FOREGROUND_WINDOW_SERVICE_ 
NT 0x00000000 

82: #define USER_GET_FORECROUND_WINDOW_SERVICE_ 
2K 0x00001189 

83: #define USER_GET_FORECROUND_WINDOW_SERVICE_ 



XP 0x00001194 
84: typedef HWND 
85: (NTAPI 

86: *NT_USER_GET_FOREGROUND_WINDOW) (); 

87: NT_USER_MESSAGE_CALL UserMessageCallHandler = N 

ULL; 

88: NT_USER_POST_MESSAGE UserPostMessageHandler = 
NULL; 

89: NT_USER_SEND_INPUT UserSendlnputHandler = NULL; 
90: HOOK.FUNCTION hUserMessageCall = {0}; 
91: HOOK_FUNCTION hUserPostMessage = {0}; 
92: HOOK.FUNCTION hUserSendlnput = {0}; 
93: // Utility stuff we need to call 

94: NT_USER_QUERY_WINDOW NtUserQueryWindow = NUL 
L; 

95: NT_USER_GET_FOREGROUND_WINDOW NtUserGetFore 

groundWindow = NULL; 

96: #define MAX_PROCESS_FILTER 4 

97: DWORD dwProcFilter[MAX_PROCESS_FILTER] = {0}; 

98: HWND hWndLast = NULL; 

99: DWORD dwProcldLast = 0; 

100: BOOL lsProcessFilter(DWORD dwProcessID) 

101: { 



102: DWORD n; 

103: for (n = 0; n < MAX_PROCESS_FILTER; n++) 
104: { 

105: if (dwProcFilter[n] == dwProcessID) 
106: return TRUE; 
107: } 

108: return FALSE; 
109: } 

110: NTSTATUS AddProcessFilter(DWORD dwProcessID) 
HI: { 

112: DWORD n; 

113: for (n = 0; n < MAX_PROCESS_FILTER; n++) 
114: { 

115: if (dwProcFilter[n] == dwProcessID) 
1 16: return STATU S.SUCC ESS; 
117: if (dwProcFilter[n] == 0) 
118: { 

119: dwProcFilter[n] = dwProcessID; 
120: return STATU S.SUCC ESS; 
121: } 
122: } 

123: return STATUS.UNSUCCESSFUL; 
124: } 



125: NTSTATUS DelProcessFilter(DWORD dwProcessID) 
126: { 

127: DWORD n; 

128: for (n = 0; n < MAX_PROCESS_FILTER; n++) 
129: { 

130: if (dwProcFilter[n] == dwProcessID) 
131: { 

132: dwProcFilter[n] =0; 
133: return STATU S.SUCC ESS; 
134: } 
135: } 

136: return STATUS.UNSUCCESSFUL; 
137: } 

138: BOOL WndContinue( HWND hWnd) 
139: { 

140: if (NtUserQueryWindow) 
141: { 

142: DWORD dwProcessID; 
143: if (hWnd == hWndLast) 
144: { 

145: dwProcessID = dwProcldLast; 
146: } 
147: else 



148: { 

149: dwProcessID = NtUserQueryWindow(hWnd, QUER 

Y_WINDOW_PROCESS); 

150: dwProcldLast = dwProcessID; 

151: hWndLast = hWnd; 

152: } 

153: if (IsProcessFilter(dwProcesslD) && 

154: (dwProcessID != GetCurrentProcesslDO)) 

155: { 

156: return FALSE; 
157: } 
158: } 

159: return TRUE; 
160: } 

161: BOOL MsgContinue( HWND hWnd, UINT Msg, WPARA 
M wParam, 

162: LPARAM IParam) 
163: { 

164: BOOL bProtect = FALSE; 
165: BOOL bContinue = TRUE; 
166: switch (Msg) 
167: { 

168: // If we're not interested in a message at all, let's f 



ind 




that out first 


169: 


//case WM.QUIT: 


170: 


case WM.TIMER: 


171: 


if (IParam == 0) 


172: 


break; 


173: 


// fall through 


174: 


case WM.KEYDOWN: 


175: 


case WM.SYSKEYDOWN: 


176: 


case WM.KEYUP: 


177: 


case WM.SYSKEYUP: 


178: 


case WM.LBUTTONDOWN: 


179: 


case WM.LBUTTONUP: 


180: 


case WM.RBUTTONDOWN: 


181: 


case WM.RBUTTONUP: 


182: 


case WM.COMMAND: 


183: 


case BM.SETSTATE: 


184: 


case BM.SETCHECK: 


185: 


case BM.CLICK: 


186: 


case WM_M BUTTON DOWN: 


187: 


case WM.MBUTTONUP: 


188: 


case WM.MBUTTONDBLCLK: 


189: 


case WM.RBUTTONDBLCLK: 



190: case WM.LBUTTONDBLCLK: 
191: return WndContinue(hWnd); 
192: } 

193: return TRUE; 
194: } 

195: NTSTATUS 
196: NTAPI 

197: HookUserMessageCalK HWND hWnd, UINT Msg, WPA 
RAM wParam, 

198: LPARAM IParam, LRESULT IResult, 

199: DWORD dwUnknownl, DWORD dwUnknown2) 

200: { 

201: NTSTATUS Status = STATU S.SUCC ESS; 

202: if (MsgContinue(hWnd, Msg, wParam, IParam) && 

UserMessageCallHandler) 

203: { 

204: Status = UserMessageCallHandler(hWnd, Msg, wPa 
ram, IParam, 

205: IResult, dwUnknownl, dwUnknown2); 

206: } 

207: #ifdef .DEBUG 
208: else 
209: { 



210: DbgPrintnGDI NtUserMessageCall] BLOCKED: " 
211: "hWnd %x, Msg %x, wParam %x, IParam %x, " 
212: "Process %x\n", hWnd, Msg, wParam, IParam, 
213: GetCurrentProcesslDO); 
214: } 

215: #endif //.DEBUG 
216: return Status; 
217: } 

218: NTSTATUS 
219: NTAPI 

220: HookUserPostMessage( HWND hWnd, UINT Msg, WPA 

RAM wParam, 

221: LPARAM IParam) 

222: { 

223: NTSTATUS Status = STATU S.SUCC ESS; 

224: if (MsgContinue(hWnd, Msg, wParam, IParam) && 

UserPostMessageHandler) 

225: { 

226: Status = UserPostMessageHandler(hWnd, Msg, wP 
aram, IParam); 
227: } 

228: #ifdef .DEBUG 
229: else 



230: { 

231: DbgPrintnGDI NtUserPostMessage] BLOCKED: " 
232: "hWnd %x, Msg %x, wParam %x, IParam %x, " 
233: "Process %x\n", hWnd, Msg, wParam, IParam, 
234: GetCurrentProcesslDO); 
235: } 

236: #endif //.DEBUG 
237: return Status; 
238: } 
239: UINT 
240: NTAPI 

241: HookUserSendlnput( UINT nlnputs, // count of inp 
ut events 

242: PVOID plnputs, // struct _LPINPUT plnputs - ar 
ray of 

input events 

243: intcbSize) // size of structure 
244: { 

245: HWND hWnd = 0; 

246: if (NtUserGetForegroundWindow && 

247: WndContinue(hWnd = NtUserGetForegroundWindo 

wO) && 

248: UserSendlnputHandler) 



249: { 

250: return UserSendlnputHandler(nlnputs, plnputs, cb 

Size); 

251: } 

252: #ifdef .DEBUG 
253: else 
254: { 

255: DbgPrint("[GDI NtUserSetlnput] BLOCKED: " 
256: "hWnd %x, Process %x\n", 
257: hWnd, GetCurrentProcesslDO); 
258: } 

259: #endif //.DEBUG 
260: return 0; 
261: } 

262: NTSTATUS StartTrackGDIO 
263: { 

264: NTSTATUS Status = STATU S.SUCC ESS; 

265: Status = Hooklnt2EService( &hUserMessageCall, 

HookUserMessageCall, 

266: Findlnt2EServiceBylD(USER_MESSAGE_CALL_SER 
VICE)); 

267: if (Status == STATUS.SUCCESS) 

268: UserMessageCallHandler = hUserMessageCall.pOl 



dFunction; 

269: Status = Hooklnt2EService( &hllserPostMessage, 
HookllserPostMessage, 

270: Findlnt2EServiceBylD(USER_POST_MESSAGE_SER 
VICE)); 

271: if (Status == STATUS.SUCCESS) 

272: UserPostMessageHandler = hUserPostMessage.pOl 

dFunction; 

273: Status = Hooklnt2EService( &hUserSendlnput, Hook 
UserSendlnput, 

274: Findlnt2EServiceBylD(USER_SEND_INPUT_SERVIC 

E»; 

275: if (Status == STATUS.SUCCESS) 

276: UserSendlnputHandler = hUserSendlnput.pOldFun 

ction; 

277: Status = Findlnt2EServiceCall( 

278: Findlnt2EServiceBylD(USER_QUERY_WINDOW_SE 

RVICE), 

279: (PVOID*)(&NtUserQueryWindow)); 

280: Status = Findlnt2EServiceCall( 

281: Findlnt2EServiceBylD(USER_GET_FOREGROUND_ 

WINDOW.SERVICE), 

282: (PVOID*)(&NtUserGetForegroundWindow)); 



283: return Status; 
284: } 

285: NTSTATUS StopTrackGDIO 

286: { 

287: NTSTATUS Status = STATU S.SUCC ESS; 

288: Status = Unhooklnt2EService(&hUserMessageCall); 

289: Status = Unhooklnt2EService(&hUserPostMessage); 

290: Status = Unhooklnt2EService(&hUserSendlnput); 

291: return Status; 

292: } 

12 ] At lines 262-284 a "StartTrackGDIO" function is an initial- 
ization routine that is analogous to the "StartTrackLPCO" 
function described above. In this case, however, the mes- 
sages that are being intercepted are GDI (graphics device 
interface) messages. As shown at line 265, one of the 
messages being intercepted is a "hUserMessageCall" 
which is the kernel equivalent of a user-level "SendMes- 
sage". A second type of message that is being trapped is a 
"hUserPostMessage" as illustrated at line 269 which is 
equivalent to a "PostMessage". A third is a "hUserSendln- 
put" message as shown at line 273 which is a way of 
pushing keystrokes, mouse movement, and mouse clicks 
into the data stream that may potentially be used to cir- 



cumvent a security application. These are only three ex- 
amples and those skilled in the art will appreciate that 
there are other messages that could also be misused and 
may be handled in a similar fashion. 

[0113] At the end of the "StartTrackGDIO" initialization routine 
there are two additional calls to a "Findlnt2EServiceCaN" 
routine at lines 277-279 (for finding "NtUserQueryWin- 
dow") and lines 280-282 (for finding "NTUserGetFore- 
groundWindow"). Direct access to these routines is not 
available in the kernel, so these routines obtain pointers 
from the dispatch table in a manner similar to that de- 
scribed above with respect to the "StartTrackLPCO" func- 
tion. The "StopTrackGDIO" function at lines 285-292 is 
analogous to the above-described "StopTrackLPCO" rou- 
tine. As shown, the three services that are hooked by the 
"StartTrackGDIO" function are unhooked by this "Stop- 
TrackGDIO" function. 

[0114] T he "IsProcessFilter", "AddProcessFilter", and "DelProcess- 
Filter" routines at lines 100-137 illustrate a very simple 
mechanism of one embodiment for testing if a process is 
a process that is of interest, registering a process by 
adding it to a registry, and removing a process from the 
registry, respectively. As shown at lines 96-97 the registry 



in this embodiment comprises an array of up to 4 pro- 
cesses, although a different number of processes could of 
course be included in the array if desired. Processes can 
be added to an empty location in the array through the 
"AddProcessFilter" routine or removed from the array "Del- 
ProcessFilter" routine. The "IsProcessFilter" routine tests if 
a process is a process of interest. Information about reg- 
istered processes can also be obtained by consulting the 
array. 

15 ] The "HookUserMessageCall" routine commencing at line 
197 looks for a potentially malicious application which 
might send messages in order to circumvent security 
measures. At line 202 a "MsgContinue" routine is called. 
The "MsgContinue routine itself is shown at lines 
161-194. For performance reasons, the "MsgContinue" 
routine checks to determine if the message is a Windows 
message that is of interest. It is easier and faster to first 
determine if the message is of interest before examining 
the message in more detail. For example, as shown at 
lines 170-172, the "IParam" (or second parameter) of a 
"WM.TIMER" message is identified as being of interest be- 
cause a malicious application can potentially insert a 
pointer as an avenue for attacking a targeted application. 



At lines 174-190 is a list of Windows messages that may 
be generated based on a number of different types of user 
interaction with the system (e.g., keystrokes, button 
clicks, mouse clicks, and so forth) and that may poten- 
tially be used to attack a protected process or application. 
If a message is one of these types of messages that may 
potentially cause harm or circumvent security, a "WndCon- 
tinue" routine is called as shown at line 191 to determine 
if the window relates to a protected process or application 
and to track this type of potentially harmful message. 
16 ] The "WndContinue" routine at lines 138-160 determines if 
the window is owned by a registered process that is being 
tracked (i.e., protected). As shown at line 140, a check is 
first made to ensure that an "NtUserQueryWindow" func- 
tion is available as it is necessary for execution of this 
"WndContinue" routine. The "NtUserQueryWindow" func- 
tion is a function available in the operating system kernel 
that provides certain characteristics about a window 
("hWnd"). Next, the code at lines 143-146 is a perfor- 
mance optimization. As shown, the last window is cached 
given that a number of messages are typically received re- 
lating to the same window. Caching the last window may 
avoid the need to look up information about the window 



with a call to the "NtUserQueryWindow" function each 
time. In the currently preferred embodiment a single win- 
dow is cached; however, multiple windows could be 
cached if desired. 

[° 117 ] If information is not available in cache, then an "NtUser- 
QueryWindow" function is called to obtain information on 
the window as show at lines 147-152. In this case the in- 
formation that is of interest is the Windows process 
("dwProcessID"). As shown at line 153, after the process 
identifier ("dwProcessID") has been obtained, the above- 
described "IsProcessFilter" routine is called to determine if 
this particular process is of interest (e.g., a registered 
process). If the process is of interest, then at line 154 the 
process identifier is also compared with "CetCurrentPro- 
cesslDO" to verify that the calling process and the process 
being called are not the same. In other words, if the call- 
ing process is calling itself, then this call is not of interest 
for purposes of controlling interprocess communications. 

[° 118 ] In the currently preferred embodiment, a determination is 
made as to whether the process that is being called is a 
protected process or application (i.e., one that has been 
registered). If the process that is being called is a pro- 
tected process then "FALSE" is returned as shown at line 



156. Otherwise, "TRUE" is returned as shown at line 159 
to indicate that the communication should be allowed to 
proceed. However, those skilled in the art will appreciate 
that other filters may also be used, if desired. For exam- 
ple, a test could be made to determine if a process was a 
"trusted" process. If the process was a trusted process it 
may be allowed to proceed (i.e., communicate). As an- 
other example, the individual parameters of a call that 
may potentially be malicious can be changed or manipu- 
lated (both before and after invocation). For example, one 
may wish to fill in a data structure with useless data in an 
effort to thwart a malicious application. In addition, the 
status that is returned can also be adjusted if desired. For 
example, the status may be adjusted to let a potentially 
malicious application think that it succeeded when, in 
fact, it is blocked. 
1Q ] If the process is not protected (i.e., if "TRUE" is returned), 
then an actual handler is called as illustrated at lines 
202-206 of the "HookUserMessageCall" routine. In this 
event the message is sent to the original destination. Oth- 
erwise, the message is blocked, and a message status is 
returned as shown at lines 208-216. A debug message is 
also printed. The same approach is also used in the 



"HookUserPostMessage" and "HookUserSendlnput" rou- 
tines at lines 220-261 to address different types of mes- 
sages that may be sent. 

[° 12 °] It should be noted that there is slight variation with the 
"HookUserSendlnput" routine. The "Sendlnput" function 
sends a message comprising a structure of various 
keystrokes or mouse events to the kernel to insert into the 
window that has the current focus. In this case, an addi- 
tional step is required when the message is received, 
which is to determine the current window that is to re- 
ceive the message by calling a "NtUserGetForegroundWin- 
dow" function as shown at lines 246-248. Otherwise, the 
same approach is used as described above for the 
"HookUserMessageCall" routine. If the message is allowed, 
the original handler is called and the message is sent to 
the original destination. Otherwise, the message is 
blocked and a zero ("0") is returned as provided at lines 
253-260. In this manner, a registered process or applica- 
tion is protected from messages (e.g., keystroke and/or 
mouse event messages) that may be sent by malicious ap- 
plications to an active window in an effort to disable or 
interfere with the protected application. 

[0121] while the invention is described in some detail with spe- 



cific reference to a single-preferred embodiment and cer- 
tain alternatives, there is no intent to limit the invention to 
that particular embodiment or those specific alternatives. 
For instance, those skilled in the art will appreciate that 
modifications may be made to the preferred embodiment 
without departing from the teachings of the present in- 
vention. 



